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FOREWORD 


The  Infantry  Forces  Research  Unit  of  the  U.S.  Army  Research  Institute  for  the 
Behavioral  and  Social  Sciences  conducts  research  to  investigate  the  challenges  presented  by  the 
21st  Century  battlefield.  Force  modernization  efforts  must  be  guided  by  research  to  identify 
those  systems  and  procedures  that  can  produce  the  greatest  benefits  for  U.S.  forces  now  and  in 
the  future. 

This  study  investigated  the  situation  awareness  (SA)  of  platoon  leaders  in  simulated 
MOUT  (Military  Operations  on  Urbanized  Terrain)  missions.  Since  information  availability  and 
usage  are  integral  to  the  development  of  situational  dominance  for  any  military  force, 
identification  of  the  multidimensional  elements  that  comprise  a  soldier’s  SA  is  a  critical  first  step 
in  developing  technologies  and  training  methods  to  improve  SA.  A  preliminary  analysis  of 
platoon  leaders’  SA  requirements  formed  the  basis  for  developing  measures  to  quantify  SA  in 
Infantry  exercises.  Three  instruments  were  developed,  providing  both  objective  and  subjective 
SA  measures  which  were  then  implemented  during  simulated  MOUT  missions. 

The  investigation  established  that  SA  could  be  quantified  and  measured  in  a  light  Infantry 
MOUT  environment,  despite  the  complexities  of  the  information  demands.  The  results 
demonstrated  the  utility  of  virtual  exercises  in  the  Squad  Synthetic  Environment  to  support  both 
research  and  training.  Not  only  were  SA  differences  identified  between  experience  levels,  but 
also  between  scenarios  and  the  point  in  the  simulation  at  which  they  were  queried,  providing  an 
initial  demonstration  of  the  sensitivity  of  the  measures.  Qualitative  and  quantitative  SA 
differences  between  more  experienced  and  less  experienced  officers  suggest  it  may  be  possible 
to  train  less  experienced  officers  to  attend  to  the  available  information  differently,  thus 
enhancing  both  SA  and  decision-making  while  improving  the  cost-effectiveness  of  leader 
training.  Critical  aspects  of  the  research  were  briefed  to  the  Chief  of  the  Dismounted  Battlespace 
Battle  Lab  (DBBL)  Simulation  Center  on  14  November  2000  and  to  the  Deputy  Director  of  the 
DBBL  on  17  April  2001. 


ZITA  M.  SIMUTIS 
Technical  Director 
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MEASURES  OF  PLATOON  LEADER  SITUATION  AWARENESS  IN  VIRTUAL 
DECISION-MAKING  EXERCISES 

EXECUTIVE  SUMMARY  _ _ _ 


Research  Requirements: 

As  the  Army  moves  to  exploit  information  age  technology.  Infantry  forces  will  be  called 
upon  to  gather  information  and  achieve  situational  dominance  in  an  increasingly  complex 
environment.  Superior  situation  awareness  (SA),  in  terms  of  the  warfighter’s  ability  to  access 
and  use  available  information  to  improve  lethality,  survivability,  and  communication,  will  be  a 
pivotal  factor  in  the  ability  of  the  Infantry  force  to  meet  this  challenge.  The  challenges  of 
Military  Operations  on  Urbanized  Terrain  (MOUT)  contribute  to  the  complexity  of  the  task  of 
Infantry  forces,  through  the  effects  of  a  three  dimensional,  non-linear  battlefield,  a  poorly 
defined  enemy,  an  unpredictable  and  volatile  civilian  presence,  and  restrictive  Rules  of 
Engagement  (ROE).  In  this  complex  environment,  the  systematic  measurement  of  SA  will 
promote  better  understanding  of  the  specific  information  requirements  of  warfighters,  and  yield 
vital  insights  regarding  the  techniques  used  by  Infantry  leaders  to  enhance  S  A,  and  ultimately 
decision-making  and  action. 

Infantry  platoon  leaders  operate  in  a  complex  environment  requiring  that  they  attend  to 
multiple  information  sources,  prioritize  among  competing  and  sometimes  conflicting  goals,  and 
make  rapid  decisions,  all  under  highly  stressful  conditions  where  the  loss  of  life,  either  their  own 
or  others’,  is  a  constant  threat.  To  complicate  the  matter,  platoon  leaders  are  often  relatively 
inexperienced  officers,  with  minimal  service  time,  training  and  experience  to  draw  on.  In  tto 
environment,  superior  S A  provides  tremendous  advantages  to  those  with  the  ability  to  acquire  it 
and  the  experience  to  use  it. 

Procedure: 

The  current  project  focused  on  three  distinct  functional  goals.  The  first  phase  involved 
an  S  A  requirements  analysis  identifying  elemental  factors  essential  for  the  development  of 
superior  S  A  by  platoon  leaders  in  a  MOUT  mission.  Infantry  subject  matter  experts  assisted  in  a 
goal-directed  task  analysis  to  identify  the  S  A  requirements.  The  second  phase  involved 
development  of  objective  and  subjective  SA  measures  to  support  research  of  Infantry  leaders’ 

SA  during  MOUT  exercises  in  the  Squad  Synthetic  Environment,  a  virtual  simulation.  The  third 
and  phase  entailed  implementation  of  these  measures  in  four  separate  simulated  MOUT 
missions  to  determine  whether  the  measures  could  reveal  S  A  differences,  both  between  scenarios 
and  within  a  scenario  at  different  times.  Perhaps  most  important,  the  virtual  investigation 
explored  differences  between  the  level  and  type  of  SA  in  experienced  and  inexperienced  platoon 
leaders. 


By  identifying  characteristic  differences  in  the  processing  of  SA  information  with 
experience,  it  may  be  possible  to  identify  training  methods  that  can  increase  the  efficiency  and 
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effectiveness  of  training  less  experienced  platoon  leaders  to  see  the  patterns  and  threats  that  are 
more  apparent  to  officers  with  higher  levels  of  experience.  This  would  provide  the  double 
benefit  of  maximizing  training  value  while  decreasing  the  training  time  required  to  produce 
officers  who  are  better  prepared  for  the  battlefield. 

Findings; 

Despite  a  small  sample  size,  data  analysis  revealed  several  findings  of  interest.  First, 
experienced  officers  were  better  at  locating  both  friendly  and  enemy  elements  on  a  map. 

Second,  more  experienced  officers  focused  their  attention  on  the  enemy  to  a  greater  degree  than 
less  experienced  officers,  so  that  they  had  a  better  understanding  of  the  enemy  strengths  and  the 
threat  posed,  while  less  experienced  officers  better  understood  friendly  strengths.  While  further 
research  is  needed,  this  finding  could  have  significance  for  developing  training  methods.  Third, 
the  objective  SA  measures  demonstrated  sensitivity  to  experience  level,  type  of  scenario,  and 
point  in  the  simulation  at  which  the  measures  were  taken,  providing  preliminary  indications  of 
their  utility  for  the  study  of  S  A  under  the  unique  conditions  encountered  in  Infantry  operations. 

The  investigation  established  that  S A  could  be  quantified  and  measured  in  a  light  Infantry 
MOUT  environment,  despite  the  complexities  of  the  information  demands.  The  results 
demonstrated  the  utility  of  virtual  exercises  in  the  Squad  Synthetic  Environment  for  both 
research  and  training.  The  accuracy  differences  identified  between  experience  levels,  scenarios 
and  measurement  halts  provide  an  initial  demonstration  of  the  sensitivity  of  the  SA  measures. 

Utilization  of  findings: 

The  S  A  measures  developed  here  can  be  used  in  follow-on  research  to  delve  more  deeply 
into  the  construct  of  SA  in  the  light  Infantry  environment.  Additional  research  can  test  these 
measures  both  in  simulations  and  in  live  training  environments.  The  finding  that  more 
experienced  officers  have  a  better  understanding  of  the  locations  of  both  enemy  elements  and 
their  own  platoon  members,  and  the  implication  of  this  information,  can  be  used  to  train  less 
experienced  officers  to  focus  on  key  information.  Further,  the  finding  that  more  experienced 
officers  are  better  at  identifying  both  the  highest  enemy  threat  and  the  strongest  enemy  location 
shows  an  increase  in  situation  comprehension  in  these  areas.  By  focusing  on  the  ways  in  which 
S A  changes  with  experience,  it  may  be  possible  to  train  less  experienced  officers  to  attend  to  the 
available  information  differently.  Trainers  could  develop  instructional  techniques  to  improve  S  A 
accordingly,  focusing  on  the  development  of  a  mindset  that  constantly  considers  what  the  enemy 
will  do  next.  Such  advances  would  enhance  both  SA  and  decision-making  while  improving  the 
cost-effectiveness  of  leader  training. 
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MEASURES  OF  PLATOON  LEADER  SITUATION  AWARENESS  IN  VIRTUAL  DECISION¬ 
MAKING  EXERCISES 


Introduction 

Situation  awareness  (SA),  knowing  what  is  going  on  in  the  situation  around  you,  is  a 
fundamental  requirement  for  combat  success.  It  forms  the  foimdation  for  military  decision¬ 
making  and  task  execution.  In  the  demanding  Infantry  combat  environment,  superior  SA  brings 
tremendous  advantages  by  promoting  information  dominance,  improving  security  and 
survivability,  and  optimizing  lethality.  The  future  battlefield  calls  for  advanced  technologies, 
leader  development,  and  training  concepts  targeted  at  enhancing  SA  at  all  echelons.  These  goals 
can  only  be  achieved  through  systematic  effort  and  knowledge  regarding  SA.  As  Infantry  forces 
apply  technological  advances  to  expand  battlefield  information  flow,  it  becomes  increasingly 
important  to  understand  the  factors  shaping  SA  in  Infantry  operations,  to  include  the 
interrelationships  among  those  factors.  Experience  has  proven  that  more  information  does  not 
necessarily  produce  better  SA  or  improve  situational  dominance.  The  development  of  training 
programs  and  technologies  that  enable  employment  of  forces  with  high  levels  of  SA  depends 
upon  a  solid  foundation  of  knowledge  regarding  SA  in  the  Infantry  arena. 

The  Challenge  of  Situation  Awareness  in  Infantry  Platoon  Operations 

Endsley,  Holder,  Leibrecht,  Garland,  Wampler  and  Matthews  (2000)  provide  an  analysis 
of  the  role  of  S  A  in  Infantry  operations.  Infantry  platoon  leaders  decide  how  they  will  deploy, 
orient,  and  direct  their  squads  based  on  their  mission,  situation  and  SA.  Achieving  high  levels  of 
SA  in  the  highly  complex  and  dynamic  environment  is  not  easy.  Many  stressors  act  to  degrade 
the  platoon  leader’s  SA,  or  to  prevent  him  from  gaining  a  high  level  of  SA  to  begin  with.  Time 
pressure  and  the  rapid  tempo  of  operations  can  significantly  challenge  platoon  leaders  who  often 
must  struggle  to  maintain  an  up-to-date  awareness  of  a  rapidly  changing  reality.  The  conditions 
for  gathering  and  assimilating  information  may  rapidly  deteriorate  during  combat  operations. 

Fatigue  brought  on  by  heavy  physical  exertion,  lack  of  sleep  and  nighttime  operations 
also  degrade  the  platoon  leader’s  ability  to  detect  and  process  information  vital  to  good  SA. 

Poor  environmental  conditions,  including  noise,  fog,  weather,  and  smoke  can  directly  obscure 
critical  information.  Stress  and  anxiety  associated  with  warfare  and  the  inherent  uncertainty  and 
confusion  can  all  act  to  reduce  SA  (Endsley  et  al.,  2000).  Periods  of  significant  task  underload 
or  task  overload  can  also  lead  to  SA  problems. 

The  factors  that  shape  SA  also  can  be  greatly  influenced  by  the  enemy,  who  can  alter  the 
tempo  of  the  battle  and  dramatically  affect  the  conditions  under  which  a  battle  is  fought.  Thus, 
Infantry  operations  frequently  must  be  conducted  under  the  challenges  of  a  number  of  factors, 
some  naturally  occurring,  some  task  or  enemy  induced,  that  can  all  act  to  seriously  degrade  SA. 

Situation  Awareness  in  Infantry  Operations 

Endsley  (1988)  formally  defined  SA  as  “the  perception  of  the  elements  in  the 
environment  within  a  volume  of  time  and  space,  the  comprehension  of  their  meaning  and  the 
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projection  of  their  status  in  the  near  future”  (p.  97).  This  mental  representation  of  the  state  of  the 
world  involves  perceiving  critical  factors  in  the  environment  (Level  1  SA),  comprehending  what 
those  factors  mean  (Level  2  SA),  particularly  when  integrated  in  relation  to  the  soldier's  goals, 
and  at  the  highest  level,  projecting  what  will  happen  in  the  near  future  (Level  3  SA).  The  higher 
levels  of  SA  allow  soldiers  to  function  in  a  timely  and  effective  manner.  These  three  levels  are 
depicted  in  Figure  1,  and  will  be  described  further. 


Figure  1.  Model  of  situation  awareness  levels  and  decision-action  process  (from  Endsley  et  al., 
2000). 

Level  1  Situation  Awareness — Perception  of  the  Elements  in  the  Environment 

Achieving  SA  begins  with  perceiving  the  status,  attributes,  and  dynamics  of  relevant 
elements  in  the  environment.  Important  elements  include  the  position  and  actions  of  friendly 
forces,  enemy  forces,  and  civilians;  terrain  features;  obstacles;  and  weather.  In  Infantry 
operations  factors  such  as  noise,  smoke,  confusion  and  the  dynamics  of  a  rapidly  changing 
situation  may  often  obscure  critical  elements  of  the  situation.  Numerous  sources  of  information 
compete  for  the  platoon  leader’s  limited  attention  and  mental  processing  resources.  The  platoon 
leader  can  obtain  information  from  direct  observation  of  the  environment,  from  verbal  and  non¬ 
verbal  communication  with  others,  or  from  electronic  systems  and  sensors.  Each  source  of 
information  may  carry  its  own  level  of  reliability.  Confidence  in  information  (based  on  past 
experience  with  the  system,  organization,  or  individual  providing  it)  forms  a  critical  part  of  Level 
1  SA  for  the  Infantry  platoon  leader. 

Level  2  Situation  Awareness — Comprehension  of  the  Current  Situation 

The  platoon  leader’s  comprehension  of  the  situation  stems  from  a  synthesis  of  disjointed 
Level  1  elements.  Level  2  SA  extends  to  an  understanding  of  the  significance  of  the  information 
that  is  present,  in  the  context  of  the  platoon  leader’s  goals.  The  platoon  leader  integrates  Level  1 
data  to  form  a  holistic  picture  of  his  environment,  including  a  comprehension  of  the  significance 
of  objects  and  events.  For  example,  upon  seeing  the  impression  of  a  certain  type  of  vehicle  track 
in  soft  ground,  an  experienced  leader  may  realize  that  enemy  units  have  passed  through  the  area 
and  consequently  adopt  a  stealthier  posture.  A  less  experienced  leader  may  see  the  same  cues 
(Level  1  SA),  but  not  be  able  to  understand  their  meaning  as  well.  The  platoon  leader  typically 
interprets  Level  1  SA  (perceived  data)  with  reference  to  his  goals  or  plans  in  order  to  build  Level 
2SA. 
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Level  3  Situation  Awareness — Projection  of  Future  Status 


At  the  highest  level  of  SA,  the  platoon  leader  is  able  to  predict  or  project  the  future 
actions  of  the  elements  in  the  environment,  at  least  for  the  very  near  term.  He  accomplishes  this 
by  combining  knowledge  of  the  status  and  dynamics  of  the  elements  with  comprehension  of  the 
situation  (Level  1  and  Level  2  S A).  Platoon  leaders  with  a  high  level  of  SA  are  able  to  anticipate 
where  and  when  the  enemy  will  strike.  They  can  estimate  how  much  time  they  have  until 
reinforcements  arrive  or  until  supporting  fires  can  be  delivered  on  a  target.  This  look-ahead 
capability  gives  them  the  knowledge  and  time  necessary  to  decide  on  the  most  favorable  course 
of  action  to  meet  their  objectives  and  goals. 

Differences  in  Abilities  Required  to  Develop  Situation  Awareness  Among  Individuals 

j 

In  addition  to  the  problems  that  the  battlefield  environment  poses  on  the  development  of 
SA,  significant  individual  differences  exist  in  the  degree  to  which  people  are  able  to  detect  and 
assimilate  information  to  form  a  coherent  and  complete  picture  of  the  situation.  While  this  issue 
has  not  been  studied  in  detail  in  Infantry  operations,  as  much  as  a  tenfold  difference  in  SA 
abilities  has  been  reported  among  trained  individuals  in  other  domains.  Anecdotal  information 
would  suggest  that  individual  differences  likely  exist  in  those  involved  in  Infantry  operations  as 
well.  These  SA  abilities  have  been  found  to  be  highly  stable  within  individuals  (Endsley  & 
Bolstad,  1994). 

A  number  of  factors  most  likely  contribute  to  individual  differences  in  SA  ability 
(Endsley  et  ah,  2000).  Some  may  involve  basic  capabilities,  such  as  pattern  matching  skills, 
perceptual  speed,  spatial  ability,  and  attention  sharing.  At  the  same  time,  significant  advantages 
in  SA  can  be  gained  through  training  and  experience  in  the  warfighting  environment.  Training 
and  experience  allow  leaders  to  develop  mental  models  and  relevant  memory  stores  that  provide 
rapid  real-time  pattern  matching  of  perceived  information  to  form  the  highest  levels  of  SA. 

These  memory  structures  make  it  possible  for  an  individual  to  understand  what  information  is 
important,  how  to  direct  one’s  attention  to  maximum  advantage,  and  how  to  correctly  interpret 
and  integrate  the  information  that  is  perceived.  Under  the  strain  of  battlefield  conditions,  these 
capabilities  are  decisive  in  allowing  Infantry  soldiers  and  leaders  to  gain  and  maintain  SA. 

Improving  the  SA  skills  of  Infantry  leaders  offers  potential  payoff  in  combat 
effectiveness.  Little  detailed  information  is  currently  available  about  how  SA  differs  among 
Infantry  leaders  as  they  gain  experience  and  knowledge,  however.  Because  SA  itself  has  not 
been  studied  in  detail  in  the  Infantry  environment,  little  is  known  about  what  factors  allow  highly 
successful  leaders  to  quickly  gain  and  maintain  high  levels  of  SA,  or  how  well  this  issue 
differentiates  those  leaders  from  others.  By  better  understanding  the  processes,  skills  and 
knowledge  associated  with  high  (and  low)  levels  of  SA  in  Infantry  operations,  training  programs 
that  are  more  carefully  focused  on  enhancing  SA  skills  can  be  created  to  help  inexperienced 
officers  more  quickly  achieve  higher  levels  of  both  SA  and  performance.  Examining  the  ways  in 
which  SA  differs  between  experienced  and  inexperienced  Infantry  officers  forms  one  goal  of  the 
present  research. 


Evaluating  Situation  Awareness  in  Infantry  Operations 


Future  research  on  the  nature  of  SA  in  Infantry  operations  and  the  development  and 
validation  of  training  programs  or  new  technologies  depends  upon  the  development  and 
validation  of  measures  of  SA  for  Infantry  operations.  “When  you  can  measure  what  you  are 
speaking  about,  and  express  it  in  numbers,  you  know  something  about  it;  but  when  you  cannot 
measure  it,  when  you  cannot  express  it  in  numbers,  your  knowledge  is  of  a  meager  and 
unsatisfactory  kind”  (Lord  Kelvin,  quoted  in  Bartlett  &  Kaplan,  1992,  p.  504). 

By  carefully  examining  the  SA  developing  behaviors  of  experienced  and  inexperienced 
Infantry  officers  and  measuring  the  level  of  SA,  we  may  be  able  to  gain  insight  into  the  ways  in 
which  warfighters  can  form  SA  under  challenging  conditions. 

Situation  Awareness  Measurement  Techniques 


Measures  of  SA  can  be  classified  into  four  broad  types  (Endsley  et  al.,  2000): 

•  Process  indices-which  examine  how  individuals  acquire  and  process  cues  available  in 
their  environment.  These  process  indices  may  be  obtained  using  devices  such  as  eye 
trackers  or  by  studying  communication  patterns, 

•  Direct  measures  of  SA-including  purely  subjective  measures  such  as  self-ratings  or 
observer  ratings,  and  more  objective  measures  of  SA  which  compare  an  individual’s 
reported  perceptions  of  the  environment  to  some  “ground  truth”  reality, 

•  Behavioral  measures  of  S A-which  try  to  infer  SA  from  the  actions  that  individuals 
choose  to  take,  based  on  the  assumption  that  good  actions  will  follow  from  good  SA 
and  vice-versa;  and 

•  Performance  measures  of  SA-which  try  to  infer  SA  from  the  end  result,  based  on  the 
assumption  that  if  more  kills  are  scored,  for  instance,  SA  must  have  been  higher. 

The  pros  and  cons  of  these  general  measurement  approaches  and  their  applicability  to 
Infantry  operations  are  outlined  in  Endsley  et  al.  (2000).  For  the  purposes  of  the  present 
research,  we  concentrated  on  developing  and  validating  more  direct  measures  of  SA.  Aside  from 
the  fact  that  other  more  inferred  measures  of  S  A  are  fairly  circular  in  logic  and  therefore  of  little 
direct  use  in  developing  an  understanding  of  S  A  in  Infantry  operations,  more  direct  measures  of 
SA  can  provide  far  more  detail  about  the  SA  construct  itself.  That  detail  is  needed  to  develop 
new  training  methods.  Direct  measures  of  SA  include  both  subjective  and  objective 
measurement  techniques. 

Objective  Measures  of  Situation  Awareness 

The  most  widely  used  approach  to  objectively  measuring  SA  is  the  Situation  Awareness 
Global  Assessment  Technique  (SAGAT)  (Endsley,  1988, 1995).  Using  SAGAT,  a  simulated 
exercise  is  halted  at  randomly  selected  times,  information  sources  (e.g.,  communication 
channels,  virtual  displays,  or  new  information  technologies)  are  blanked  and  the  exercise  is 
suspended  while  participants  quickly  answer  questions  about  their  current  perceptions  and 
understanding  of  the  situation.  Participant  perceptions  can  then  be  compared  to  the  real  situation 
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(as  determined  by  simulation  computer  data  or  experienced  observers  who  have  knowledge 
about  the  actual  state  of  events)  to  provide  an  objective  measure  of  SA.  By  collecting  SA  data  in 
this  manner,  SAGAT  provides  an  objective,  unbiased  assessment  of  SA.  This  method  provides  a 
tool  capable  of  measuring  SA  across  all  three  levels,  based  on  a  comprehensive  analysis  of 
domain-specific  SA  requirements  (Endsley  et  al.,  2000). 

In  other  domains  such  as  piloting  high  performance  aircraft,  SAGAT  has  regularly 
yielded  a  high  degree  of  validity  (Endsley  et  al.,  2000).  It  has  good  predictive  validity,  with 
SAGAT  scores  indicative  of  pilot  performance  in  a  combat  simulation  (Endsley,  1990a). 

Content  validity  was  also  established,  showing  the  SAGAT  queries  to  be  relevant  to  SA  in  a 
fighter  aircraft  domain  (Endsley,  1990b).  A  number  of  studies  have  demonstrated  no  noticeable 
impact  on  performance  associated  with  inserting  the  freezes  to  collect  S A  data  via  SAGAT 
(Endsley,  1995,  2000).  New  queries  (detailing  each  of  the  critical  aspects  of  the  situation  that 
the  Infantry  leader  or  soldier  can  be  asked  to  report  upon  during  a  “halt”)  need  to  be  established 
for  the  Infantry  domain,  however,  to  make  this  approach  viable  for  studying  SA  in  Infantrymen. 

Subjective  Measures  of  Situation  Awareness 


An  easy  to  administer  method  for  measuring  SA  is  to  ask  exercise  participants  to  provide  a 
rating  of  their  own  SA.  Alternatively,  skilled  observers  may  be  asked  to  provide  a  rating  of  the 
SA  possessed  by  the  exercise  participants.  While  subjective  ratings  are  simple  and  direct,  they 
also  have  several  limitations.  Individuals  making  subjective  assessments  of  their  own  SA  have  no 
objective  basis  for  their  judgments.  They  may  not  know  what  they  do  not  know  (e.g.,  there  may 
be  an  enemy  just  over  the  next  hill  waiting  to  ambush  them).  Such  subjective  measures  also  tend 
to  be  global  in  nature,  and  as  such  do  not  provide  the  detailed  diagnostic  resolution  that  is 
available  with  objective  measures. 

Subjective  self-assessments  of  SA  have  been  found  to  be  poorly  related  to  objective 
measures  of  SA  (Endsley  &  Selcon,  1997).  Nonetheless,  a  person’s  subjective  level  of  SA  may 
be  important  for  determining  how  he  will  act.  Those  with  the  belief  that  they  have  good  SA  may 
be  more  likely  to  act  upon  that  knowledge  (to  either  good  or  ill  effect,  depending  on  the  accuracy 
of  that  belief),  and  those  who  believe  their  SA  to  be  poor  may  be  less  likely  to  act  (either  wisely 
or  unwisely)  (Christ,  McKeever,  &  Huff,  1994).  The  Situation  Awareness  Rating  Technique 
(SART),  the  most  commonly  used  subjective  measure  of  SA,  also  incorporates  workload 
assessments  that  are  inappropriate  for  the  Infantry  domain.  Therefore,  we  sought  to  develop  a 
simple  subjective  self-rating  measure  of  SA  for  this  effort. 

Observer  ratings  may  be  slightly  better  than  self-ratings  because  more  information  about 
the  true  state  of  the  environment  is  usually  available  to  the  observer  than  to  the  mission 
participants.  However,  observers  tend  to  have  less  insight  into  the  mental  state  of  the  person 
being  evaluated  and  are  forced  to  rely  more  on  observable  actions  and  verbalizations  of  the 
participants  to  infer  the  person’s  level  of  SA.  One  means  for  minimizing  this  difficulty  is  to  ask 
observers  to  evaluate  the  degree  to  which  individuals  are  carrying  out  actions  and  exhibiting 
behaviors  that  should  promote  the  achievement  of  high  levels  of  S  A.  This  approach  removes 
some  of  the  subjectivity  associated  with  making  judgments  about  someone’s  internal  state  of 
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knowledge  and  allows  them  to  make  judgments  about  things  that  are  more  readily  observable.  In 
the  current  effort  we  sought  to  develop  a  behaviorally  anchored  rating  scale  for  this  purpose. 


Study  Objectives 

The  present  research  is  an  exploratory  effort  conducted  within  a  virtual  battlefield 
simulation — ^the  Squad  Synthetic  Environment  (SSE) — at  the  platoon  level.  The  research  had 
several  main  objectives: 

1 .  To  develop  an  understanding  of  the  specific  factors  that  are  important  for  SA  at  the 
level  of  the  platoon  leader, 

2.  To  develop  objective  and  subjective  measures  for  studying  SA  in  platoon  leaders  and 
to  conduct  a  preliminary  evaluation  of  their  utility  and  validity,  and 

3.  To  explore  the  quantitative  and  qualitative  ways  in  which  SA  differs  between 
experienced  and  inexperienced  platoon  leaders  using  these  measures. 

These  objectives  were  accomplished  in  three  phases: 

•  Phase  1 :  Situation  Awareness  Requirements  Analysis 

•  Phase  2:  Situation  Awareness  Measures  Development 

•  Phase  3:  SoIdier-in-the-Loop  Investigation. 

Phase  1:  Situation  Awareness  Requirements  Analysis 

Overview 

The  SA  requirements  analysis  was  performed  as  a  goal-directed  task  analysis  following 
the  methodology  of  Endsley  (1993)  and  Endsley  and  Rodgers  (1994).  The  SA  requirements 
were  defined  as  those  dynamic  information  needs  associated  with  the  major  goals  or  sub-goals  of 
the  platoon  leader  in  performing  his  job  in  a  MOUT  (Military  Operations  on  Urbanized  Terrain) 
mission.  To  accomplish  the  analysis,  the  major  primary  and  secondary  goals  of  the  mission  were 
identified,  along  with  the  major  subgoals  necessary  for  meeting  each  goal.  Associated  with  each 
subgoal,  the  major  decisions  facing  the  platoon  leader  were  identified.  The  SA  requirements  for 
making  these  decisions  and  carrying  out  each  subgoal  were  then  identified.  These  requirements 
focused  not  only  on  what  information  the  platoon  leader  needs,  but  also  on  how  that  information 
is  integrated  or  combined  to  support  each  decision. 

Several  considerations  need  to  be  mentioned  in  relation  to  the  SA  requirements  analysis: 

1 .  At  any  given  time  more  than  one  goal  or  subgoal  may  be  operational,  although  they 
may  not  have  the  same  priority.  The  analysis  does  not  assume  any  prioritization 
among  the  goals,  or  that  each  subgoal  within  a  goal  will  always  be  active. 

2.  The  analysis  was  based  on  goals  or  objectives,  not  tasks.  The  analysts  strove  to  be  as 
technology-free  as  possible.  How  the  information  is  acquired  was  not  addressed.  It 
could  be  through  direct  observation,  verbal  communication,  new  technology,  or 
cognitive  processing  by  the  platoon  leader  of  new  or  previously  acquired  information. 
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Many  of  the  higher-level  SA  requirements  fall  into  the  latter  category.  The  way  in 
which  information  is  acquired  can  vary  widely  between  individuals,  over  time,  and 
between  situations. 

3.  The  analysis  sought  to  determine  what  platoon  leaders  would  ideally  like  to  know  to 
meet  each  goal.  It  was  recognized  that  they  often  must  operate  on  the  basis  of 
incomplete  information  and  that  some  desired  information  may  not  be  available  at  all. 

4.  Static  knowledge,  such  as  doctrine,  procedures  or  rules  of  engagement  (ROE),  was 
outside  the  bounds  of  this  analysis.  The  method  focused  only  on  dynamic  situational 
information  affecting  what  the  platoon  leader  does. 

Method 


Source  of  Input 

Six  male  Infantry  subject  matter  experts  (SMEs),  three  active  duty  and  three  retired 
commissioned  officers,  assisted  in  initial  knowledge  elicitation  sessions  using  the  goal-directed 
task  analysis  methodology  (Endsley,  1993;  Endsley  &  Rodgers,  1994).  The  SMEs  participated 
in  one  intensive  individual  interview  session  each,  lasting  approximately  two  hours.  One  of  the 
six,  an  active  duty  field  grade  officer,  collaborated  in  a  final  review  of  the  goal  hierarchy 
developed  from  earlier  sessions. 

Procedure 


Each  SME  responded  to  a  series  of  open-ended  questions  by  the  authors  designed  to  elicit 
detailed  responses  concerning  doctrinally-based  goals  and  the  decisions  associated  with  the 
accomplishment  of  those  goals.  The  SME  then  addressed  the  information  needed  to  formulate 
these  decisions.  Interviewers  held  their  questions  until  the  SME  reached  a  natural  stop  in  his 
narrative  response,  so  as  not  to  interrupt  the  flow  of  ideas.  At  this  time,  particular  care  was  taken 
to  ascertain  exactly  how  each  piece  of  information  was  used.  Higher-level  assessments  related  to 
comprehension  and  projection  were  determined  in  this  manner. 

The  six  initial  interviews  were  scheduled  two  per  day,  with  at  least  a  one  week  interval 
between  interview  dates  to  allow  interviewers  to  review  the  data,  break  it  down  and  reorganize  it, 
identifying  knowledge  gaps  to  be  filled  in  during  the  next  scheduled  sessions.  After  the  first  two 
complete  interview  sessions,  SMEs  viewed  a  graphical  representation  of  a  preliminary  goal 
hierarchy  developed  from  data  collected  in  the  early  sessions,  coupled  with  a  review  of  the 
available  literature  in  the  field  of  Infantry  SA.  SMEs  examined  the  outline  with  a  view  to 
developing  consensus  and  identifying  gaps  in  the  analysis.  This  process  continued  through  three 
iterations,  including  review  of  the  first  draft  of  the  completed  document,  until  SMEs  were  in 
general  agreement  with  the  analysis.  A  field  grade  SME  with  research  experience  performed  a 
final  review  and  refinement  of  the  completed  goal  hierarchy. 

Results 


The  detailed  results  of  the  complete  goal-directed  task  analysis  can  be  found  in  Appendix 
A.  Figure  2  shows  the  overall  goal  hierarchy  structure  of  the  analysis,  with  Attack,  Secure  and 
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Hold  Terrain  shown  as  the  overarching  objective.  While  this  is  not  the  precise  overarching 
objective  of  every  MOUT  mission,  an  overwhelming  majority  of  mission  goals  would  fit  beneath 
that  umbrella.  One  of  the  main  thrusts  of  this  research  was  to  develop  SA  requirements  and 
measures  applicable  across  a  broad  range  of  operational  parameters. 

The  overarching  objective  (Attack,  Secure  and  Hold  Terrain)  was  partitioned  into  seven 
primary  goals,  where  the  priority  of  individual  goals  varies  across  missions  and  even  across  the 
course  of  a  mission.  The  seven  primary  goals  are: 

•  Avoid  Casualties 

•  Negate  Enemy  Threat 

•  Movement:  Reach  Point  X  by  Time  Y 

•  Assault  Through  Objective 

•  Hold  Objective 

•  Provide  Stability  and  Support  Operations  (SASO) 

•  Function  in  a  Team  Environment 

Listed  under  each  of  these  seven  primary  goals  are  the  secondary  goals,  which  are  often 
employed  to  meet  the  mission  objectives.  Each  secondary  goal  shown  in  Figure  2  is  listed  on  a 
separate  page  in  the  detailed  results  of  the  requirements  analysis  (Appendix  A),  broken  down 
into  subgoals.  For  each  subgoal,  some  of  the  questions  that  the  platoon  leader  is  considering  are 
listed,  followed  by  the  SA  elements  necessary  to  answer  these  questions. 

To  facilitate  a  discussion  of  the  requirements  analysis  format,  Figme  3  illustrates  the  SA 
requirements  for  secondary  goal  1.1 — ^Avoid  enemy  detection.  This  secondary  goal  is 
partitioned  into  three  subgoals :  1.1.1  — Project  enemy  behavior,  1 . 1 .2 — ^Avoid  danger  areas,  and 
1 . 1 .3 — Utilize  available  cover  and  concealment.  All  three  of  these  subgoals  might  come  into 
play  during  the  operation  of  other  goals,  and  they  might  occur  at  times  when  the  secondary  goal 
here.  Avoid  enemy  detection,  is  either  impossible  or  undesirable.  For  instance,  if  the  enemy 
already  knows  the  platoon’s  location,  the  platoon  leader  will  still  want  to  project  what  the 
enemy’s  behavior  might  be,  avoid  danger  areas,  and  utilize  any  available  cover  and  concealment. 
Thus,  when  these  subgoals  are  listed  in  later  parts  of  the  analysis,  they  are  listed  under  the 
secondary  goal  of  interest,  assigned  a  new  number  connecting  the  subgoal  to  the  secondary  goal, 
but  the  questions  and  SA  requirements  are  not  repeated.  The  original  identification  number  for 
the  subgoal  is  used  to  refer  the  reader  to  these  listings  of  questions  and  SA  requirements.  For 
example,  under  secondary  goal  5.1 — ^Prepare  for  enemy  counterattack,  is  a  listing  for  subgoal 
5.1.5 — Project  enemy  behavior  (1.1.1),  showing  the  reference  back  to  the  secondary  goal  where 
this  subgoal  is  delineated. 
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Figure  2.  SA  requirements  analysis:  primary  goal  structure. 


As  shown  in  Figure  3,  the  bottom  box  in  each  column  lists  the  areas  of  SA  needed  to 
answer  the  questions  and  meet  the  goal  in  the  boxes  above  it.  If  a  complete  set  of  SA 
requirements  from  another  goal  is  needed  to  meet  the  current  goal,  the  original  goal  is  listed  in 
the  SA  requirements  in  bold  font,  which  refers  the  reader  to  a  complete  listing  of  SA 
requirements. 


What  is  the  most  likely  Course  of  Action 
(COA)  for  the  enemy?? 

What  is  the  most  dangerous  COA  for  the 
enemy? 


What  is  the  least  exposed  position  or 
avenue  of  approach? 

Can  I  avoid  danger  areas? 

Do  I  have  time  to  avoid  danger  areas? 


How  can  cover  and  concealment  be 
utilized  within  existing  operational 
constraints? 


_ J. _ 

project  enemy  posture 
enemy  vulnerabilities 
enemy  actions 

enemy  strengths/weaknesses 
probability  of  enemy  contact 
areas  of  cover  &  concealment 

Project  enemy  behavior  (1.1.1) 

Terrain/obstacles 
area  of  operation 
time  constraints 
time  available 

own  vulnerabilities  time  required  for  task/movement 

areas  of  cover  &  concealment 
exposure  areas 
enemy  LP/OP  locations 
Terrain 

Friendly  situation 
Enemy  threat 


exposure  areas 
enemy  LP/OP  locations 
Terrain 

Friendly  situation 
Enemy  threat 


Project  enemy  behavior  (1.1.1) 

time  constraints 
time  available 

time  required  for  task/movement 
Areas  of  own  vulnerability 
areas  of  concealment 
exposure  areas 
enemy  LP/OP  locations 
Terrain 

Enemy  Threat 


Figures.  Example  from  S  A  requirements  analysis:  Secondary  goal  1.1 — ^A  void  enemy 
detection. 


Many  SA  elements  come  into  play  in  very  diverse  settings  and  goals,  and  to  list  them 
individually  under  each  subgoal  would  be  redundant.  Instead,  these  often  required  information 
requirements  are  listed  separately  in  categories  on  the  final  page  of  the  analysis.  Where  these 
items  are  called  upon  in  other  pages  of  the  analysis,  they  are  also  shown  in  bold  letters.  For 
example,  certain  elements  of  the  Friendly  Situation  play  into  virtually  every  decision  made  by 
the  platoon  leader,  such  as  the  Commander’s  Intent  and  the  unit  mission. 

Figure  4  summarizes  the  secondary  SA  elements  identified  for  the  platoon  leader  in  the 
analysis  across  all  of  the  major  goals  and  subgoals  in  the  MOOT  mission.  Level  3  (projection) 

S  A  items  are  shown  flush  with  the  left  edge  of  the  cell,  while  Level  2  (comprehension)  elements 
are  indented  once,  and  Level  1  (perception)  items  are  indented  twice. 
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FRIENDLY  SITUATION 


ENEMY  THREAT 

Anticipated  enemy  actions 
project  enemy  posture 
enemy  vulnerabilities 
resistance  expected 
projected  impact  on  mission 
projected  dispersion  around  weapons 
projected  impact  of  LP/OP  location 
enemy  expectations  of  my  behavior 
projected  enemy  casualties 
likelihood  of  attack 
enemy  level  of  resistance 
enemy  combat  power 
projected  enemy  breach  points 
likely  enemy  avenues  of  approach 
enemy  maneuverability 
enemy  strengths 
enemy  weaknesses 
enemy  disposition 
enemy  intent 
enemy  objective 
enemy  composition 
enemy  actions 

enemy  strengths/weaknesses 
probability  of  enemy  contact 
areas  of  cover  &  concealment 
exposure  areas 
enemy  LP/OP  locations 
enemy  psychology 
enemy  patterns  of  movement 
enemy  field  of  observation 
enemy  field  of  fire 
location 
dispersion 
numbers 
weapons 
ammo/supplies 
objective 
unit  type 
equipment 
experience  level 
morale/commitment 
vehicle 

capabilities/skills/training 
recent  actions 
enemy  doctrine 
past  behavior/actions 
religious/poliiical  beliefs 
locations  of  ammo/supplies 
movement  of  weapons 
security/patrol  formations  &  schedule 
enemy  center  of  gravity 
building  vantage  points 


CIVILIAN  SITUATION 

projected  civilian  behavior 
projected  civilian  casualties 
projected  civilian  response 
level  of  threat 
potential  for  goodwill 
potential  for  escalation 
level  of  organization 
mood  of  crowd 
religious/political  beliefs 
morale/commitment 
intent 

proximity  to  sensitive  sites  (church,  school, 
political  center,  etc) 
location 
number 

agitators  present 
threatening  actions 
weapons 
training/skills 

Rules  of  Engagement 


:  Anticipated  friendly  actions 
Troop  psychology 
'  Projected  troop  fatigue 
i  Soldier  mental  fatigue 
•  Projected  performance  on  task 
Potential  for  fratricide 
Anticipated  relative  locations  of  other  units 
!  Own  vulnerabilities 
projected  casualties 
I  strategic  significance  of  weapons 
■  projected  impact  of  action  on  mission 
i  projected  effect  of  action 
;  projected  benefit/cost  of  action 
:  projected  ability  to  avoid  fires 
:  likelihood  of  deception  success 
projected  effects  of  moving  casualties 
!  ability  to  detect  enemy  presence 
ability  to  avoid  enemy  detection 
likelihood  of  enemy  contact 
dispersion  required 
forces  required 
resources  required 
force  ratio 

,  Commander’s  Intent 

'  unit  objective 

planned  friendly  actions 
combat  power 
availability  of  fires 
priority  of  fires 
combat  multipliers  available 
timing  of  support 
protection  provided 
friendly  composition 
unit  type 
experience  level 
morale/commitment 
character  of  soldiers 
individual  soldier  discipline 
capabilities/skills/training 
fatigue/soldier  load 
troop  disposition 
distance  traveled 
time  in  lead/on  task 
resistance  encountered 
troop  doctrine 
past  behavior/actions 
retigious/political  beliefs 
friendly  field  of  observation 
friendly  field  of  fire 
enemy  LP/OP  locations 
supply  level 
re-supply  access 
weapons  characteristics 
Specifics  of  pre-planned  fires 
Confidence  in  intelligence  information 
'  communication  plan 

reliability  of  comm  channel 
Information  to  communicate 
areas  of  poor  communication 
handling  of  casualties 
medical  facilities/personnel 
evacuation  plan 

number  of  casualties 

severity  of  casualties 

commo  call  signs  and  frequencies 

equipment 

technologies  available  (NVG,  laser,  etc) 
vehicles 
skills 
training 
experience 
reliability 
>  fatigue 

location 
dispersion 
!  numbers 

:  weapons 

ammo/supplies 
Range  to  other  troops 
I  direction  to  other  troop 

j  timing  of  fires 

fires  calling  procedure 
location  of  fires 
troop  movement 


TERRAIN 

Projected  impact  of  terrain 
■  Effects  of  terrain  on  communications 
.  Estimated  time  required  for  movement 
.  Projected  safety  of  routes 
Projected  physical  requirements  of  route 
'  Projected  mental  requirements  of  route 
cover  &  concealment 
:  area  of  operations 

type  of  terrain 
routes  of  ingress  &  egress 
fortifications 
day/night 
features 
high  ground 
construction  type 
conditions 

building  accessibility 
building  usage 

mechanical  entry  points 
explosive  entry  points 
rubble 
mud 

booby  traps 

areas  of  damage 

severity  of  damage 

light  level  available 

building  vantage  points 

windows 

doonvays 

rooftops 

vegetation 

obstacles 

buildings 

size/type  of  room 

artivities  in  adjacent  rooms 

usage 

mass  construction 
framed  construction 
entry  points 
vantage  points 


WEATHER  CONDITIONS 

projected  effects  of  weather 
temperature 
precipitation 
wind 

direction 
speed 
light  levels 
visibility 
ambient  noise 


TIME  CONSIDERATIONS 

:  Projected  time  to  objective 
Projected  difficulty  of  task 
Projected  time  to  cover  &  concealment 
Time  constraints 
time  available 

time  required  for  task/movement 


DANGER  AREAS 

Projected  areas  of  enemy  attack 
Projected  areas  of  enemy  detection 
roads 
streams 
open  areas 
railroad  tracks 
corridors 
stairwells 
windows 
doorways 
funnel  areas 


Figure  4.  SA  requirements  for  Infantry  platoon  leaders  in  a  MOUT  mission. 
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Phase  2:  Situation  Awareness  Measures  Development 


This  project  endeavored  to  develop  both  objective  and  subjective  measures  of  SA  to 
investigate  SA  for  the  Infantry  platoon  leader.  Based  on  the  results  of  the  SA  requirements 
analysis,  three  instruments  were  developed  to  measure  SA  during  Infantry  platoon  leader 
operations:  SAGAT,  the  Situation  Awareness  Behaviorally  Anchored  Rating  Scale  (SABARS), 
and  the  Post-Trial  Participant  Subjective  SA  Questionnaire  (PSAQ). 

Situation  Awareness  Global  Assessment  Technique  (SAGAT) 

The  Infantry  platoon  leader  version  of  SAGAT  utilizes  a  customizable  PC-based 
computer  program  which  presents  officers  with  the  same  21  standard  queries  which  are  designed 
to  assess  all  three  levels  of  SA  (perception,  comprehension,  and  projection).  The  queries  are 
presented  at  discrete  points  in  time  during  simulated  missions.  Queries  address  major  SA 
elements  such  as  location  of  strongest  and  weakest  enemies  and  friendlies,  number  of  casualties 
suffered  by  the  officer’s  platoon,  and  expected  enemy  and  civilian  actions  over  the  next  five 
minutes. 

SAGAT  questions  were  developed  by  examination  of  the  SA  requirements  list.  Queries 
were  selected  based  on  several  criteria: 

1 .  The  query  must  be  answerable  at  any  time  during  the  mission, 

2.  It  must  have  a  right  or  mong  answer, 

3.  Participants  must  be  able  to  easily  respond  to  the  question  (Yes-No,  Multiple  Choice 
type  queries), 

4.  Queries  must  address  SA  elements,  not  strategies  and  decisions  which  build  from  SA, 

5.  Questions  must  cover  all  three  levels  of  SA,  perception,  comprehension  and 
projection. 

The  SAGAT  queries  were  selected  to  cover  the  broadest  range  of  SA  requirements 
possible,  across  the  multiple  goal  areas  that  were  assessable  in  the  simulation  environment.  An 
objective  was  to  create  the  minimum  number  of  queries  that  would  cover  the  full  range  of 
information  requirements.  The  queries  were  developed  to  assess  SA  at  a  reportable  level  that 
could  be  scored  as  correct  or  incorrect  based  on  simulator  or  SME  input.  The  queries  were 
reviewed  by  the  SMEs  for  understandability  and  appropriateness,  and  revisions  were  made 
accordingly. 

This  process  resulted  in  21  questions.  Table  1  contains  a  complete  listing  of  SAGAT 
queries,  along  with  the  available  responses.  (Eight  of  the  items  were  not  administered  in  the 
investigative  portion  of  the  study — Phase  3 — due  to  limitations  of  the  simulation  or  missions. 
Those  items  are  marked  with  an  asterisk  in  Table  1.)  The  SAGAT  method  permits  flexible 
administration  by  allowing  the  test  administrator  to  determine  which  questions  to  block,  which 
questions  to  always  show  (in  random  order),  and  which  questions  to  show  in  random  order  after 
those  questions  that  are  always  presented.  Each  query  was  presented  graphically  on  the  monitor 
of  a  computer.  The  actual  screen  image  for  each  query  is  shown  in  Appendix  B,  along  with  the 
written  instructions. 
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Table  1 


SAGAT  Queries 


_ 

Query 

Response  Options 

■ 

Indicate  the  location(s)  of  each  element  on  the 
map. 

Enemies,  Enemy  Heavy  Weapons,  Myself,  Squad  1,  Squad 
2,  Squad  3,  Weapons  Squad,  Detached  Troops,  Other 
Friendlies,  Civilians,  Commander 

2 

Which  enemy  element  is  your  highest  level 
threat? 

El-20  (enemies),  Wl-20  (enemy  heavy  weapons) 

Can  all  the  assigned  squad  tasks  be 
accomplished  within  the  time  requirements? 

Yes,  No 

4 

Which  enemy  locations  are  the  weakest? 

El -20  (enemies),  Wl-20  (enemy  heavy  weapons) 

5 

Which  enemy  locations  are  the  strongest? 

El -20  (enemies),  Wl-20  (enemy  heavy  weapons) 

6 

Which  friendly  locations  are  the  weakest? 

M,  Squad  1 ,  Squad  2,  Squad  3,  W,  D 1  - 1 0,  F 1  - 1 0 

7 

^^ich  friendly  locations  are  the  strongest? 

M,  Squad  1,  Squad  2,  Squad  3,  W,  Dl-10,  Fl-10 

8* 

Will  weather  be  an  impact  on  operations? 

Yes,  No 

Which  friendly  forces  are  currently  exposed  to 
enemy  fire/attack? 

M,  Squad  1 ,  Squad  2,  Squad  3,  W,  D 1  - 1 0,  F 1  - 1 0 

10* 

Is  fatigue  impacting  on  friendly  troops? 

Yes,  No 

11* 

Which  of  the  following  supplies  are  insufficient 
for  mission  completion? 

Water,  Ammo,  Food,  Equipment,  Troops,  None 

12 

Does  the  enemy  know  the  location  of  your 
platoon? 

Yes,  No 

Which  of  the  following  assets  are  available  to 
support  you? 

Supporting  fire,  Smoke,  NVG,  Reinforcements, 

Emergency  medical  care.  None 

14* 

"^ich  troops  are  in  locations  that  do  NOT  offer 
concealment? 

Squad  1,  Squad  2,  Squad  3,  Weapons  Squad,  Detached 
Troops,  None 

15* 

Which  troops  are  in  locations  that  do  NOT  offer 
cover? 

Squad  1,  Squad  2,  Squad  3,  Weapons  Squad,  Detached 
Troops,  None 

16 

How  many  casualties  have  you  suffered? 

0,  1,2,  3, 4,  5,  6,7,  ...,29,30,  >30 

17 

What  do  you  expect  the  enemy  to  do  in  the  next 
five  minutes? 

Attack,  Nothing,  Move  positions.  Defend,  Retreat,  Other 

18 

What  do  you  expect  civilians  to  do  in  the  next 
five  minutes? 

Become  hostile,  Riot/attack,  Form  a  crowd.  Disperse, 
Nothing,  Move  positions,  Get  in  the  way,  Other 

19 

Who  has  the  advantage  in  the  current  situation? 

Friendly  troops,  Enemy  troops,  Friendly  and  Enemy  troops 
equal 

20* 

For  which  friendly  element  are  plans  not  being 
executed  as  per  orders? 

Squad  1,  Squad  2,  Squad  3,  Weapons  Squad,  Other 

Platoons,  Supporting  units,  None 

21 

Which  friendly  elements  are  NOT  in 
communication  with  you? 

Squad  1,  Squad  2,  Squad  3,  Weapons  Squad,  Other 

Platoons,  Supporting  units.  None 

*  Not  used  in  the  soldier-in-the-loop  investigation,  Phase  3  of  this  study. 


Figure  5  shows  the  first  query  always  presented  to  the  participant  and  the  foundation  for 
all  subsequent  queries  that  showed  cartographic  data.  For  this  first  item,  participants  identified 
the  locations  of  friendly,  enemy  and  civilian  elements  on  a  map  of  the  virtual  environment. 

Small  color-coded  icons  appeared  in  a  panel  on  the  right  side  of  the  screen  and  participants  used 
a  standard  drag-and-drop  technique  to  place  items  on  the  map  or  to  move  them  from  one  location 
to  another  once  they  have  been  “dropped”  onto  the  map.  Some  elements  permitted  the  creation 
of  multiple  reports,  such  as  Enemies,  Enemy  Heavy  Weapons,  AdjacenfrOther  Friendly  Troops, 
Detached  Troops  and  Civilians.  Icons  for  other  elements  were  removed  once  they  were  placed 
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onto  the  map.  The  participant  was  only  allowed  to  locate  one  Myself,  Squad  1,  Squad  2,  Squad 
3,  Weapons  Squad,  and  Commander.  When  the  participant  was  satisfied  that  he  had  placed  all 
elements  onto  the  map  in  their  correct  location,  he  selected  “Done”  in  the  bottom  right  hand 
comer,  and  the  next  query  was  presented.  All  subsequent  queries  were  presented  in  random 
order,  to  prevent  participants  from  anticipating  the  queries  and  thus  “preparing”  for  the  SA 
assessment  by  paying  special  attention  to  those  elements  covered  in  the  SAGAT  queries.  (Note: 
If  this  type  of  selective  attention  process  had  occurred,  officers  would  be  expected  to  perform 
better  on  later  halts  than  on  earlier  halts.  The  data  analysis  showed  no  such  trend.) 


BEttS 


\  'rit.’t 


- 

"'■^'-'7-- - '  /  /'  /  / 


/  / 


Done 


Figure  5.  SAGAT  query  1. 

Situation  Awareness  Behaviorally  Anchored  Rating  Scale  (SABARS) 

The  SABARS  measures  consisted  of  28  questions  selected  from  the  question  pool 
developed  during  Phase  1  and  judged  by  an  SME  familiar  with  all  aspects  of  the  project  to  have 
relevance  for  the  SSE  simulation.  The  final  questionnaire  was  reviewed  by  all  six  SMEs  for 
understandability  and  revised  as  needed.  The  SABARS  items  elicited  ratings  fi:om  an 
independent  Observer/Controller  (0/C)  on  how  well  the  platoon  leader  exhibited  behaviors 
consistent  with  acquiring  and  disseminating  SA  information  during  the  exercise.  Since  SA 
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actually  refers  to  an  individual’s  internal  representations  of  elements  in  the  environment 
(perceptions,  comprehension,  and  projections),  it  is  important  to  note  that  the  SABARs  measure 
does  not  rate  actual  SA,  but  rather  outward  actions  that  indicate  a  greater  likelihood  of  good 
internal  representations.  While  behaviors  can  be  an  important  indicator  of  mental  processes, 
they  can  also  be  misleading. 

The  0/C’s  subjective  rating  of  the  participant’s  SA-related  behaviors  was  assessed  using 
items  such  as  “Solicits  information  from  squad  leaders,  ”  “Communicates  key  information  to 
commander,  ”  “Asks  for  pertinent  intelligence  information,  "  “Identifies  critical  mission  tasks  to 
squad  leaders,  ’’  and  “Assesses  key  finds  and  unusual  events.  ”  The  quality  of  the  behaviors  were 
rated  on  a  five-point  scale  ranging  from  1  =  “Very  Poor”  to  5  =  “Very  Good,”  with  intermediate 
ratings  of  “Poor,”  “Borderline,”  and  “Good.”  An  additional  response  of  “Not  Applicable”  was 
added  to  the  scale  for  behaviors  that  were  either  not  demonstrated  or  could  not  be  assessed  from 
the  scenario.  The  SABARS  instrument  is  shown  in  Appendix  C. 

Post-Trial  Participant  Subjective  Situation  Awareness  Questionnaire  (PSAQ) 


The  PSAQ  instrument  (presented  in  Appendix  D)  consisted  of  three  items  designed  to 
elicit  the  participant’s  subjective  ratings  for: 

•  workload — how  hard  the  officer  worked  during  the  scenario, 

•  performance — how  well  the  officer  performed  during  the  scenario,  and 

•  self-perceived  SA — how  aware  the  officer  was  of  the  evolving  situation. 

These  items  were  rated  on  a  five-point  scale.  For  workload,  a  rating  of  1  meant  that  the 
officer  was  not  working  hard,  while  5  meant  he  was  working  extremely  hard.  For  performance, 
a  low  rating  reflected  poor  performance,  while  a  high  rating  reflected  extremely  good 
performance.  A  rating  of  1  on  SA  indicated  that  the  officer  was  not  aware  of  the  evolving 
situation,  while  a  rating  of  5  indicated  the  officer  was  completely  aware  of  the  evolving  situation. 

Phase  3:  Soldier-in-the-Loop  Investigation 

Conducted  in  an  immersive  virtual  simulation  environment,  the  purpose  of  the  third 
phase  of  the  study  was  four-fold:  (a)  examine  the  suitability  of  the  SA  instruments  developed 
during  Phase  2;  (b)  explore  how  experienced  officers  might  differ  from  inexperienced  officers  in 
terms  of  S  A;  (c)  explore  the  effects  of  different  scenarios  and  measurement  halts  on  SA;  and  (d) 
examine  the  interrelationships  among  the  various  SA  measures  used. 

Method 


Participants 

Fourteen  officers  (13  Infantry  and  one  Armor)  participated  in  the  investigation.  The 
average  age  of  the  seven  lieutenants  was  23.7  years.  For  the  captains,  the  average  age  was  27.9 
years.  Time  in  service  ranged  from  1 1  to  83  months  for  lieutenants  and  49  to  133  months  for 
captains.  All  captains  had  served  as  platoon  leaders  while  none  of  the  lieutenants  had  served  in 
that  position.  All  officers  were  Airborne  qualified  and  four  from  each  group  had  completed 
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Ranger  school.  Six  lieutenants  and  three  captains  had  engaged  in  training  exercises  at  the 
McKenna  MOUT  site  at  Fort  Benning,  but  only  one  officer  had  ever  been  in  an  SSE  simulator 
like  that  used  in  this  experiment  (Pleban,  Eakin,  Salter,  &  Matthews,  2001). 

Materials 


Biographical  Information  Questionnaire.  The  Biographical  Information  Questionnaire 
(presented  in  Appendix  E)  was  a  paper-and-pencil  instrument  designed  to  elicit  general 
biographical  information  from  the  participant  such  as  age,  prior  military  experience  and  training 

SAGAT.  A  laptop  computer  was  used  to  administer  S AGAT  during  three  halts  in  each  of 
two  scenarios:  Assault  and  Defend.  Queries  were  presented  either  until  the  participant 
completed  the  full  battery,  or  until  4  minutes  had  elapsed,  whichever  came  first,  at  which  time 
the  platoon  leader  returned  to  the  combat  simulation.  Of  the  21  queries  in  the  SAGAT  battery, 

13  were  deemed  appropriate  for  the  scenarios  used  and  the  capabilities  of  the  SSE  (see  Table  1, 
where  the  omitted  items  are  denoted  by  asterisks). 

SABARS.  Immediately  following  completion  of  each  of  the  four  scenarios,  the  0/C,  a 
retired  Infantry  officer,  completed  a  SABARS  rating  form  based  on  the  participant’s  behaviors. 

It  should  be  noted  here  that  eight  items  were  removed  from  the  analysis  due  to  a  large  number  of 
“Not  Applicable”  ratings.  These  eight  items  are  marked  with  an  asterisk  in  the  SABARS  rating 
form  found  in  Appendix  C. 

PSAQ.  The  PSAQ  was  administered  at  the  conclusion  of  each  of  the  four  scenarios,  and 
participants  rated  each  item  along  a  five-point  scale.  Response  categories  varied  for  each 
question  (see  Appendix  D).  Officer  comments  were  solicited  at  the  end  of  each  item. 

Small  Unit  Leader  Decision-Making  Scenarios 

Scenarios  were  set  in  a  small  European-style  town,  a  virtual  representation  of  the 
McKenna  MOUT  training  site  at  Fort  Benning,  Georgia.  Infantry  SMEs  developed  six  possible 
scenarios;  four  were  selected  for  the  variety  and  training  value  provided.  The  four  scenarios 
were  Assault,  Defend,  SASO  and  Secure  Village/Downed  Helicopter.  In  each  scenario,  between 
four  and  seven  decision  points  occurred,  where  specific  decision-making  actions  were  expected, 
e.g.,  notify  commander  of  movement,  determine  status  of  injured  soldiers,  warn  troops  against 
firing  on  civilians.  Each  platoon  leader  participant  completed  all  four  scenarios,  but  only  the 
Defend  and  Assault  scenarios  were  used  for  SAGAT  measurements.  See  Appendices  F  and  G 
for  flow  charts  delineating  the  action  sequence  and  decision  points  for  each  SAGAT  scenario. 

Simulated  scenarios  focused  on  the  interactions  between  the  participant,  who  filled  the 
role  of  platoon  leader,  and  his  Commanding  Officer  (CO),  three  squad  leaders,  and  platoon 
sergeant.  Scenario  developers  presented  cognitively  challenging  simulations  through  the  use  of 
severe  time  constraints,  realistic  and  complex  situations,  incomplete  or  uncertain  information, 
and  multiple  decisions.  A  wide  variety  of  events  were  utilized,  e.g.,  sniper  fire,  injured 
soldiers/civilians,  NBC  (Nuclear,  Biological,  and  Chemical)  threat,  death  of  CO.  Scenario 
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instructions  required  minimal  movement  from  the  participants  to  maintain  the  focus  on  the 
decision-making  aspects  of  the  scenarios. 

Simulation  Environment 

Three  full-immersion  SSE  simulators  from  the  Dismounted  Battlespace  Battle  Lab’s 
Land  Warrior  Test  Bed  at  Fort  Benning,  Georgia,  were  employed;  one  for  the  platoon  leader  and 
two  for  role-playing  squad  leaders.  The  CO  used  a  joystick-controlled  desktop  version  of  the 
SSE  immersible  system.  Each  SSE  simulator  consisted  of  an  enclosure  with  black,  sound- 
dampening  fabric  on  three  sides  and  a  9-ft  by  8-ft  projection  screen  as  the  fourth  wall.  The 
enclosures  reduced  extraneous  light  and  minimized  distractions  from  outside  sources.  One  side 
of  the  rear  panel  fabric  in  the  participant’s  enclosure  was  tied  back  to  permit  observation  by 
those  recording  the  results  of  the  research.  Using  a  communication  system  similar  to  those  used 
in  the  field,  participants  were  able  to  communicate  with  squad  leaders,  the  CO,  and  the  platoon 
sergeant.  For  a  more  complete  description  of  the  SSE  simulation,  see  Pleban  et  al.  (2001). 

Procedure 


Each  individual  platoon  leader  participated  in  the  simulation  on  one  day.  The  order  of 
participants  (experienced  vs  inexperienced  platoon  leaders)  was  alternated  across  days,  to  guard 
against  any  confounding  effects  of  improvements  in  training  processes  or  trainers  over 
successive  days.  (For  a  more  complete  description  of  the  procedures  used,  see  Pleban  et  al. 
(2001).) 

Participant  training.  A  different  participant  served  as  platoon  leader  each  day.  The 
participant  arrived  early  in  the  morning  for  an  initial  briefing  on  the  research  objectives.  After 
completion  of  the  Biographical  Information  Questionnaire,  the  SAGAT  test  administrator 
presented  the  participant  with  written  instructions  on  the  SAGAT  procedures  (Appendix  B),  then 
led  the  participant  through  a  trial  SAGAT  run  on  the  laptop  computer. 

Following  SAGAT  training,  the  participant  entered  the  simulator  for  a  brief  training 
session,  with  instruction  on  some  of  the  key  features.  In  addition,  he  was  shown  how  various 
entities  were  depicted  in  the  virtual  environment,  i.e.,  enemies,  civilians,  and  vehicles. 

Experimental  procedure.  At  the  completion  of  training,  the  participant  received  written 
personal  profiles  of  the  CO,  the  three  squad  leaders,  and  the  platoon  sergeant.  These  profiles 
provided  cues  to  guide  the  platoon  leader  in  determining  which  squads  to  deploy  to  meet  various 
mission  objectives.  The  role-player  CO  then  met  with  the  platoon  leader  for  a  mission  briefing, 
followed  by  the  opportunity  for  questions  and  10-15  minutes  to  develop  a  plan  before  briefing 
the  squad  leaders  and  platoon  sergeant  on  the  plan.  At  this  time,  the  platoon  leader  and  role- 
players  proceeded  to  their  assigned  positions,  performed  system  checks,  and  began  the  exercise. 

Participants  encountered  either  the  SASO  or  Downed  Helicopter/Secure  Village  scenario 
first,  as  the  pre-test  condition,  followed  by  Assault,  then  Defend,  then  and  the  remaining 
scenario.  During  each  scenario,  a  researcher  recorded  data  regarding  the  decisions  made  by  the 
platoon  leader  while  the  0/C  supplied  limited  direction  to  the  participant  as  needed.  If  the 
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participant  failed  to  perform  key  actions,  he  was  prompted  accordingly.  At  the  conclusion  of 
each  scenario,  the  0/C  provided  feedback  to  the  participant  on  actions  not  performed  or  incorrect 
information  relayed.  Next,  the  participant  completed  the  PSAQ  while  the  0/C  completed  the 
SABARS  instmment.  Each  scenario  ended  with  an  After  Action  Review  (AAR)  guided  by  the 
CO. 


SAGAT.  Administration  of  SAGAT  occurred  during  the  Assault  (2"^*)  and  Defend  (3’’'*) 
scenarios.  When  the  action  was  halted  at  three  predetermined  halt  points  in  each  scenario,  the 
SAGAT  administrator  wheeled  a  cart  holding  a  laptop  computer  over  to  the  participant.  Each 
SAGAT  session  lasted  a  maximum  of  4  minutes,  less  if  the  participant  completed  the  questions 
before  the  allotted  time  expired.  The  timer  began  as  soon  as  the  participant  clicked  the  “Start” 
button  (see  Appendix  B).  For  identification  purposes,  the  participant’s  name  along  with  the 
scenario  and  halt  number  were  displayed  at  the  top  of  the  start  screen.  Query  1,  asking  the 
participant  to  locate  all  elements  on  a  map,  was  always  presented  first.  After  the  participant 
completed  locating  elements  on  the  map,  subsequent  queries  were  presented  in  random  order. 
While  the  participant  completed  the  SAGAT  queries,  the  role-player  CO  completed  a  paper  and 
pencil  version  of  the  SAGAT  queries  which  was  then  used  to  score  the  participant’s  responses. 
At  the  conclusion  of  each  SAGAT  halt,  the  laptop  was  removed  and  the  simulation  resumed 
exactly  where  it  had  been  halted. 

Responses  for  each  question  were  captured  in  a  data  file  by  the  SAGAT  program. 
Separate  files  were  created  for  each  participant  and  each  SAGAT  Scenario.  A  technical  problem 
resulted  in  loss  of  data  for  the  first  participant  in  the  Assault  Scenario.  Each  line  of  data  in  the 
file  contained  the  participant  number,  scenario,  trial  number,  halt  number,  query  number,  and  the 
participant’s  response. 

The  SAGAT  program  stored  X  and  Y  coordinates  for  each  element  in  Query  1  along  with 
a  unique  label  identifying  the  item.  For  example,  the  fifth  enemy  location  identified  would  be 
labeled  E5.  Another  program  read  these  coordinates  from  the  data  file  and  positioned  the 
elements  on  a  map.  This  map  was  used  to  score  both  location  data  and  subsequent  questions 
about  the  relative  strength  and  weakness  of  friendly  and  enemy  troops.  For  an  example  of  a 
completed  map,  see  Figure  6. 

To  score  location  data,  the  participant-generated  map  was  compared  with  (a)  a  screen 
shot  from  the  simulation’s  Plan  View  Display  showing  the  actual  locations  of  the  computer¬ 
generated  squads  engaged  in  the  exercise,  (b)  the  key  provided  by  the  CO,  and  (c)  the  briefing 
materials  given  to  participant  before  each  scenario.  Analysis  was  limited  by  the  fact  that 
enemies  and  adjacent  fiiendly  platoons  were  not  actually  depicted  by  the  computer  simulation,  so 
they  had  no  physical  location  in  the  virtual  world  even  though  the  results  of  their  actions  were 
seen  and  heard  by  the  participant.  As  a  result,  some  latitude  was  allowed  in  the  identification  of 
correct  locations  for  these  elements  of  the  simulation,  giving  the  benefit  of  the  doubt  to  the 
participant.  Missing  data  were  omitted  from  the  analyses. 
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Figure  6.  SAGAT  query  1  showing  elements  located  on  the  map  by  a  participant. 


Results 


The  various  S  A  measures  were  examined  to  determine  the  impact  of  Experience  Level 
and  Scenario  Type  on  platoon  leader  SA  during  the  exercises.  The  results  for  each  SA  measure 
are  presented  separately.  Subsequently,  the  relationship  among  the  SA  measures  is  discussed. 

Objective  Ratings  of  Situation  Awareness 

The  platoon  leader’s  responses  to  each  SAGAT  query  were  compared  to  the  actual  state 
of  the  environment  at  the  time  of  the  halt  as  recorded  by  the  simulation  computer  and  the  trained 
SME/observer.  This  process  provided  an  accuracy  score  for  each  query.  The  first  query 
provided  during  each  halt  asked  participants  to  report  on  the  location  of  their  own  platoon 
elements,  adjacent  friendly  elements,  and  enemy  elements.  Due  to  the  highly  scripted  nature  of 
the  scenarios  and  the  timing  of  the  halts,  adjacent  friendly  units  and  enemy  elements  occupied 
consistent  locations  for  a  given  scenario  and  halt  for  all  study  participants.  Thus,  if  a  sniper  fired 
shots  from  building  P-2  during  the  Assault  scenario  for  the  first  participant,  he  did  the  same  for 
every  other  participant.  This  facilitated  identification  of  the  number  of  items  that  participants 
should  have  located  on  the  map  for  a  given  halt.  The  percentages  of  elements  that  participants 
correctly  located  in  each  of  these  three  categories  (own,  adjacent,  and  enemy  elements),  along 
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with  the  total  percentage  of  correctly  identified  elements,  were  calculated.  Accuracy  for  the 
remaining  queries  was  calculated,  based  on  the  data  collected  from  the  simulator  computer  and 
the  SME/observer. 

The  accuracy  scores  for  each  SAG  AT  query  type  were  analyzed  via  Analysis  of  Variance 
(ANOVA)  to  determine  whether  they  displayed  sensitivity  to  differences  in  Experience, 

Scenario,  and  Halt  Number,  and  the  interactions  among  these  variables.  (An  arc  tan 
transformation  was  applied  to  the  S AGAT  scores  prior  to  analysis  in  order  to  meet  the 
assumptions  of  homogeneity  of  variance.)  An  overall  SAGAT  score  was  computed,  however, 
past  studies  have  shown  that  SA  accuracy  on  individual  queries  is  highly  independent,  leading  a 
combined  SAGAT  measure  to  be  insensitive  (i.e.,  it  tends  to  mask  trade-offs  in  SA  between 
different  aspects  of  the  situation).  The  results  of  this  study  were  no  different.  As  expected,  the 
overall  SAGAT  score  showed  no  significant  relationship  with  Experience,  Scenario  or  Halt 
Number.  Therefore,  each  query  was  analyzed  individually  to  determine  sensitivity  to 
experience,  scenario  and  halt  effects,  and  the  interactions  among  them.  Results  of  the  ANOVAs 
for  each  SAGAT  query  type  are  shown  in  Appendix  H  (Tables  H-1  through  H-42),  along  with 
the  means  for  each  condition  combination.  Only  the  results  of  statistically  significant  differences 
(a  =  .05)  will  be  discussed. 

Experience.  Two  of  the  four  location  items  showed  significant  differences  between 
experience  levels.  The  more  experienced  participants  were  significantly  better  at  locating  both 
enemy  troops  (p  =  .03)  and  elements  of  their  own  platoon  (p  =  .03)  on  the  map.  Three  additional 
SAGAT  queries  exhibited  significant  differences  for  experience  level,  with  more  experienced 
officers  better  at  identifying  both  the  location  of  the  strongest  enemy  (p  <  .01)  and  the  location  of 
the  element  posing  the  highest  threat  to  their  platoon  (p  =  .01)  than  less  experienced  officers. 
Conversely,  less  experienced  participants  were  better  at  identifying  locations  of  the  strongest 
friendly  elements  than  more  experienced  officers  (p  =  .01).  Figure  7  shows  these  significant 
experience  level  differences.  No  other  SAGAT  queries  produced  significant  differences  between 
experience  levels. 
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Figure  7.  SAGAT  mezin  scores  by  experience  level. 
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Scenario.  Several  queries  showed  a  significant  effect  for  Scenario  with  participants 
scoring  higher  in  the  Assault  scenario  in  identifying  the  strongest  friendly  element  (p  <  .001), 
determining  whether  enemies  knew  the  participant’s  location  (p  =  .02),  and  the  degree  to  which 
they  predicted  future  enemy  actions  (p  <  .001).  In  addition,  participants  had  higher  SA  in  the 
Assault  scenario  regarding  the  locations  of  adjacent  friendly  units  (p  <  .01)  and  total  elements 
correctly  located  overall  (p  <  .001).  Higher  SA  scores  were  found  in  the  Defend  scenario  for 
identifying  the  locations  of  the  weakest  enemy  elements  (p  <  .001)  and  exposed  friendly 
elements  (p  <  .001),  and  knowing  the  number  of  casualties  suffered  (p  <  .001).  In  the  Defend 
scenario,  however,  no  casualties  were  suffered  at  least  until  the  third  SAGAT  halt,  while  in  the 
Assault  scenario,  the  officer’s  platoon  suffered  casualties  even  before  the  first  SAGAT  halt 
occurred.  Similarly,  there  was  no  close  or  clearly  defined  threat  until  the  third  halt  of  the  Defend 
scenario,  and  therefore  no  opportunity  for  friendly  troops  to  be  exposed  to  enemy  fire.  Thus,  the 
number  of  casualties  was  zero  and  enemies  did  nothing  for  most  of  the  Defend  scenario,  making 
those  much  easier  questions  to  answer  in  this  case.  The  observed  differences  may  also  reflect  the 
fact  that  the  Assault  scenario  always  preceded  the  Defend  scenario.  Figure  8  depicts  the 
significant  differences  between  scenarios. 
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Figure  8.  SAGAT  mean  scores  by  scenario. 


Halt  number.  Three  SAGAT  queries  showed  a  significant  effect  for  SAGAT  Halt 
Number:  expected  enemy  actions  (p  <  .001),  expected  civilian  actions  (p  <  .01),  and  own 
location  known  by  enemy  (see  Figure  9).  For  these  three  queries,  performance  declined  after  the 
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first  halt,  possibly  indicating  that  S  A  on  these  factors  declined  as  the  scenario  progressed  and 
pre-mission  briefing  information  became  more  dated.  It  may  also  reflect  the  complexity  and 
uncertainty  that  are  characteristic  of  these  missions.  These  results  do  discount  the  possibility  that 
participants  may  have  been  preparing  for  the  SAGAT  queries,  as  they  did  not  improve  with 
practice,  in  agreement  with  previous  research  on  this  technique. 
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Figure  9.  SAGAT  mean  scores  by  halt  number. 

Interaction  Effects.  Finally,  four  significant  interactions  were  identified:  a  scenario  by 
halt  interaction  for  identification  of  strongest  enemy  (p  =  .01),  own  location  known  by  enemy 
(p  =  .04),  expected  enemy  actions  (p  <  .001),  and  expected  civilian  actions  (p  =  .03)  (see  Figure 
1 0).  Knowledge  of  the  strongest  enemy  location  improved  over  time  in  the  Defend  Scenario, 
while  it  declined  over  time  in  the  Assault  Scenario.  For  expected  enemy  actions,  SA  started  out 
much  higher  in  the  Assault  scenario  than  in  the  Defend  scenario,  possibly  due  to  elapsed  time 
from  the  pre-mission  briefing.  Knowledge  of  civilian  actions  started  out  higher  in  the  Defend 
Scenario,  but  decreased  more  than  in  the  Assault  Scenario.  Knowledge  of  whether  the  enemy 
was  aware  of  the  platoon  location  decreased  at  the  end  of  the  Defend  Scenario,  but  not  the 
Assault  Scenario.  These  findings  could  also  be  an  artifact  of  scenario  structure,  and  must  be 
viewed  with  caution. 

There  was  one  significant  Experience  Level  by  Scenario  Interaction  regarding  knowledge 
of  communications  with  friendly  elements  (p  =  .02)  with  experienced  platoon  leaders  being  more 
aware  of  this  in  the  defend  scenario  than  the  assault  scenario  and  novice  platoon  leaders  showing 
the  opposite  pattern  (see  Figure  1 1). 
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Figure  10.  SAG  AT  mean  scores-scenario  by  halt  interactions. 
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Figure  1 1 .  Knowledge  of  communications  with  friendly  elements  by  experience  level  and 
scenario. 

Subjective  Observer  Ratings  of  Situation  Awareness  on  SABARS 


Twenty  SABARS  items  were  completed  by  the  0/C  for  each  scenario.  Scores  for  these 
items  were  subjected  to  a  factor  analysis.  The  analysis  resulted  in  combining  the  20  SABARS 
items  into  four  factors  (see  Table  H-33),  which  account  for  approximately  67%  of  the  variance  in 
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the  items  (see  Table  H-34).  The  factors  are  shown  in  Table  2.  Two  additional  factors  did  not 
load  heavily  on  any  SABARS  items  and  were  of  questionable  value  (contributing  only  an 
additional  12%).  In  view  of  the  small  sample  size,  those  two  factors  were  dropped  from  the 
analysis.  It  should  be  noted  that  many  of  the  items  were  highly  intercorrelated  (Table  H-35). 

Table  2 

SABARS  Factors 


Factor  1: 

Gathering  Information 
and  Following 
Procedures 

Factor  2: 

Focusing  Externally 
versus  Internally 

Factor  3: 

Proactively  Seeking 
Key  Information 

Factor  4: 

Focusing  on  the  Big 
Picture 

Uses  assets  to  effectively 
assess  environment 

Identifies  critical  mission 
tasks  to  squad  leaders  (-) 

Employs  squads  tactically 
to  gather  needed 
information 

Communicates  key 
information  to  squad 
leaders 

Utilizes  a  standard 
reporting  procedure 

Solicits  information  from 
squad  leaders(-) 

Discerns  key  information 
from  reports  received 

Communicates  to  squads 
overall  situation  and 
Commander  *s  intent 

Identifies  OCOKA 
elements 

Communicates  key 
information  to  commander 

Ensures  avenues  of 
approach  are  covered 

Solicits  information  from 
commanders 

Sets  appropriate  levels  of 
alert 

Gathers  follow  up 
information  when  needed 

Locates  self  at  vantage 
point  to  observe  main 
effort 

Monitors  company  net 

Assesses  information 
received 

Asks  for  pertinent 
intelligence  information 

Assesses  key  finds  and 
unusual  events 

Asks  for  pertinent 
intelligence  information 

Gathers  follow  up 
information  when  needed 

Assesses  key  finds  and 
unusual  events 

Overall  situation 
awareness  rating 

Communicates  key 
information  to  commander 

Monitors  company  net 

Discerns  key  information 
from  reports  received 

Solicits  information  from 
commanders 

Overall  situation 
awareness  rating 

Assesses  information 
received 

Assesses  key  finds  and 
unusual  events 

Identifies  OCOKA 
elements 

Note:  Loadings  are  positive  unless  otherwise  indicated  by  a  negative  sign. 


Factor  1,  with  all  positive  factor  loadings  as  seen  in  Table  2,  is  labeled  Gathering 
Information/Following  Procedures.  The  highest  loadings  for  Factor  2  were  in  a  negative 
direction  for  Identifies  critical  mission  tasks  to  squad  leaders,  and  Solicits  information  from 
squad  leaders,  leading  experimenters  to  refer  to  this  factor  as  Focusing  Externally  versus 
Internally,  where  Internally  refers  to  the  leader’s  platoon,  and  Externally  refers  to  resources 
outside  the  platoon.  Factor  3  is  called  Proactively  Seeking  Key  Information  because  the 
information  is  actively  sought  and  the  focus  is  on  key  information.  Factor  4,  centered  on 
communicating  key  information,  is  called  Focusing  on  the  Big  Picture.  Factor  scores  were 
computed  based  on  the  weighted  combination  of  all  items  with  a  factor  loading  of  higher  than 
0.30  on  each  factor. 

These  four  SABARS  factors  were  analyzed  by  ANOVA  (Appendix  H,  Tables  H-36 
through  39)  for  all  four  scenarios  to  determine  whether  the  behaviors  differed  significantly 
across  Experience  Level  or  Scenario,  or  if  there  was  an  Experience  Level  by  Scenario 
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Interaction.  The  ANOVAs  for  the  SABARS  factor  scores  showed  two  significant  effects.  Factor 
1 — Gathering  Information/Following  Procedures — showed  significant  effect  (p  =  .05)  for 
Experience  Level,  with  more  experienced  officers  receiving  higher  scores,  as  shown  in  Figure 
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Figure  12.  SABARS  scores  for  gathering  information/following  procedures  by  experience  level. 

Factor  4 — Focusing  on  the  big  picture  also  showed  a  significant  effect  (p  =  .01)  for 
Experience  Level,  with  more  experienced  officers  receiving  higher  scores,  as  shown  in  Figure 
13. 


Table  13.  SABARS  scores  for  focusing  on  the  big  picture  by  experience  level. 
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Subjective  Self-Ratings  of  Situation  Awareness 


Participants’  PSAQ  ratings  of  Workload,  Performance  and  SA  were  analyzed  via 
ANOVA  (Appendix  H,  Tables  H-40  through  42)  for  all  four  scenarios,  but  no  significant  effects 
were  found  for  Experience  Level,  Scenario  or  an  Experience  Level  by  Scenario  Interaction. 

Comparison  of  Situation  Awareness  Measures 


Although  SAGAT  has  been  well  validated  in  many  studies  in  a  variety  of  environments, 
the  SABARS  measure  is  a  relatively  new  measure  of  SA.  To  determine  whether  the  behaviors  as 
rated  on  the  SABARS  questionnaire  or  the  PSAQ  self-ratings  are  predictive  of  the  level  of  SA 
measured  via  SAGAT,  step-wise  regression  analyses  were  conducted.  With  each  of  the  SAGAT 
queries  as  dependent  variables,  regressions  examined  whether  any  of  the  SABARS  or  PSAQ 
measures  predicted  SAGAT  performance  (on  the  same  scenarios  for  the  same  participant). 

Generally  none  or  only  one  item  showed  up  in  most  of  the  regression  models  as  being 
predictive  of  SA  as  measured  by  SAGAT.  This  may  reflect  low  predictiveness  of  some  of  the 
SABARS  measures,  or  that  SABARS  and  SAGAT  are  tapping  into  unrelated  aspects  of  SA.  It 
may  also  reflect  the  small  sample  size  of  the  investigation. 

As  shown  in  Table  3,  SABARS  factors  and  PSAQ  measures  accounted  for  a  significant 
portion  of  the  variance  in  five  SAGAT  queries.  Factor  4,  Focusing  on  the  Big  Picture,  explains 
3 1 .1%  of  the  variance  in  percentage  of  enemy  locations  correctly  identified,  while  PSAQ 
Workload  explains  15.1%  of  the  variance  in  identifying  the  locations  of  adjacent  friendly  units. 
Factor  3,  Seeking  Key  Information,  accounts  for  21 .6%  of  the  variance  in  identifying  the 
location  of  the  strongest  friendly  force.  Two  items,  SABARS  Factor  3,  Seeking  Key 
Information,  and  the  PSAQ  measure  of  SA  explain  41.3%  of  the  variance  in  the  participants’ 
ability  to  correctly  identify  the  number  of  casualties  suffered.  Finally,  PSAQ  Workload 
comprises  22.9%  of  the  variance  in  the  degree  to  which  participants  were  aware  of  which  force 
had  the  advantage.  Table  3  shows  these  significant  relationships,  with  their  associated  F-values, 
probability  values  and  the  squared  correlation  (R^)  values,  corresponding  to  the  portion  of 
variance  in  the  dependent  variable  (SAGAT  query)  accounted  for  by  the  model  (SABARS  and 
PSAQ  measures). 

One  caveat  must  be  added  here:  some  of  these  SA  measures  are  highly  inter-correlated. 
(See  Table  H-35,  showing  the  correlation  matrix  for  all  SAGAT  measures  with  significant 
correlations  highlighted.)  Inter-correlation  of  variables  can  have  significant  implications  for  the 
validity  of  the  regression  model  (for  example,  only  one  of  two  highly  correlated  variables  might 
be  included).  Further  research  is  needed  to  confirm  these  results. 
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Table  3 


Summary  of  SAGAT  Regression  Analysis  with  SABARS  and  PSAQ  Ratings 


SAGAT  Query 

Model 

F-Value 

P-Value 

Enemy  Locations 

4-Big  Picture  Focus 

11.293 

0.003 

0.311 

Adjacent  Friendly  Locations 

Self-Rated  Workload 

4.442 

0.045 

0.151 

Strongest  Friendly 

3-Seeks  Key  Info 

6.871 

0.015 

0.216 

U  Casualties 

3-Seeks  Key  Info, 
Self-Rated  SA 

8.434 

0.002 

0.413 

Advantage 

Self-Rated  Workload 

7.429 

0.012 

0.229 

Discussion 


These  findings  from  the  soldier-in-the-loop  investigation  show  an  interesting  effect  of 
experience  on  platoon  leader  SA.  While  more  experienced  officers  demonstrated  superior  Level 
1  SA  in  identifying  the  locations  of  both  their  ovra  troops  and  enemy  troops — as  would  be 
expected,  the  more  important  story  involves  the  subsequent  cognitive  processes — ^the 
transformation  of  the  information  into  higher-level  SA.  More  experienced  officers  identified  the 
strongest  enemy  and  the  highest  enemy  threat  with  greater  accuracy  than  officers  with  less 
experience,  while  less  experienced  officers  demonstrated  superior  performance  at  identifying  the 
strongest  friendly  elements.  Thus,  not  only  did  experienced  leaders  demonstrate  higher  levels  of 
SA  on  certain  factors,  as  might  be  expected,  but  SA  also  proved  qualitatively  different  with  level 
of  experience.  That  is,  with  increasing  levels  of  experience,  platoon  leaders  shift  their  focus 
from  concentrating  on  friendly  disposition  to  focusing  more  on  enemy  disposition.  In  support  of 
this  finding,  Shattuck,  Graham,  Merlo  and  Hah  (2000)  conducted  a  card  sort  procedure  in  which 
they  found  that  novices  initially  requested  more  information  on  friendlies  than  enemies,  but 
experienced  officers  tended  to  prefer  enemy  information  to  friendly  information.  This  shift  in 
information  attended  to  with  the  gaining  of  experience  merits  careful  investigation.  Training 
programs  that  teach  new  officers  which  information  is  most  important,  and  how  to  assess  and 
assimilate  enemy  information  into  their  mental  picture,  may  be  warranted  as  the  Army 
increasingly  strives  to  develop  more  cost-effective  methods  for  training  new  officers. 

In  addition  to  examining  the  SA  of  platoon  leaders,  this  research  represents  an  initial 
effort  to  systematically  develop  a  SAGAT  battery  for  Infantry  operations.  While  the  present 
research  was  not  designed  to  validate  the  measures,  per  se,  it  provides  a  good  indication  of  their 
sensitivity  and  utility  for  assessing  SA  in  Infantry  operations.  The  SAGAT  measures  showed 
good  sensitivity  to  differences  in  platoon  leader  experience  level  and  scenario  and  demonstrated 
how  SA  changed  significantly  over  the  course  of  the  simulation  exercises.  Given  that  SA  is  a 
multi-dimensional  construct,  it  is  also  important  that  SAGAT  showed  good  diagnosticity  in 
revealing  the  ways  in  which  SA  changed  across  experience  levels  and  scenarios.  Overall,  the 
investigation  indicates  that  further  use  and  development  of  the  SAGAT  measure  in  Infantry 
exercises  would  be  warranted. 
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The  SABARS  instrument  represents  an  initial  attempt  to  develop  a  subjective  scale  of 
situation  assessment  behaviors  that  might  be  useful  for  evaluating  SA  in  Infantry  operations. 
Ratings  indicated  that  more  experienced  officers  exhibited  more  behaviors  related  to  Gathering 
Information  and  Following  Procedures  (Factor  1)  than  less  experienced  officers,  as  might  be 
expected.  Because  the  rater  completing  the  SABARS  questions  knew  which  officers  were 
experienced  and  which  were  not,  however,  it  is  possible  that  this  merely  reflected  a  halo  effect. 

Two  SABARS  factors.  Proactively  Seeking  Key  Information  and  Focusing  on  the  Big 
Picture,  were  significantly  predictive  of  the  level  of  SA  exhibited  by  the  officers  on  several 
SAGAT  queries.  Behaviors  associated  with  Focusing  on  the  Big  Picture  accounted  for  almost 
one-third  of  the  variance  in  the  platoon  leader’s  knowledge  of  enemy  locations.  Results 
indicating  that  more  experienced  officers  focus  greater  attention  on  enemy  disposition  suggest 
that  training  new  officers  in  the  behaviors  associated  with  Focusing  on  the  Big  Picture  may  be 
effective  in  increasing  SA  among  less  experienced  officers.  Proactively  Seeking  Key 
Information  demonstrated  significant  predictive  value  for  the  officers’  knowledge  of  the 
strongest  friendly  location  and  the  number  of  own  casualties. 

It  should  be  noted  that  this  was  an  initial  attempt  to  look  at  the  utility  of  a  measure  such 
as  SABARS.  Since  it  was  collected  with  only  one  rater,  no  analysis  can  be  made  of  its 
reliability.  In  addition,  only  a  small  sample  size  was  available  in  this  research.  Its  results  must 
therefore  be  caveated  accordingly.  More  research  is  needed  to  assess  both  the  reliability  and 
validity  of  this  measure. 

Since  this  research  was  based  on  a  small  sample,  it  is  difficult  to  say  whether  these 
findings  will  hold  up  to  further  investigation  or  whether  additional  relationships  between  the 
SABARS  factors  and  the  level  of  SA  possessed  by  platoon  leaders  might  exist.  Therefore,  no 
serious  changes  to  SABARS  are  recommended  at  this  time.  Rather,  these  findings  can  only  be 
taken  to  indicate  a  tentative  level  of  utility  for  such  a  measure.  Further  research  is  needed  to 
examine  the  SABARS  scales  with  a  larger  test  population  and  a  wider  range  of  scenarios  and 
testing  conditions.  Also,  it  should  be  noted  that  the  relationships  found  are  modest,  though 
significant.  Research  is  needed  to  further  determine  significant  sources  of  SA  differences  in 
addition  to  the  behaviors  incorporated  on  this  scale.  The  PSAQ  scales  were  not  sensitive  to  the 
experience  levels  or  scenarios  used  in  this  research. 

While  evidence  for  the  sensitivity  of  global  measures  of  self-rated  SA  and  workload  such 
as  those  used  here  has  been  mixed,  PSAQ  ratings  did  show  some  sensitivity  in  the  present 
research.  Self-rated  SA  was  significantly  related  to  the  officers’  knowledge  of  the  number  of 
casualties.  Self-rated  workload  was  significantly  related  to  the  officer’s  knowledge  of  the 
location  of  adjacent  fiiendlies  and  which  side  had  the  advantage.  Workload  has  been  found  to  be 
significantly  related  to  low  SA  in  other  domains,  and  under  certain  conditions  can  certainly  be 
seen  to  be  a  problem  for  SA  in  Infantry  operations. 

Finally,  as  a  part  of  this  research,  a  significant  effort  was  made  to  delineate  the  SA 
requirements  of  platoon  leaders.  This  analysis  should  be  seen  as  preliminary  and  should  be 
further  developed  and  validated  with  Infantry  officers  who  are  experienced  in  MOUT  operations. 
In  addition,  similar  analyses  are  needed  for  other  Infantry  missions  and  echelons.  In  addition  to 
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providing  the  basis  for  the  development  of  the  SA  measures  used  in  this  research,  these  analyses 
of  goals,  decisions  and  SA  requirements  are  highly  valuable  in  developing  new  technologies  and 
training  methods  for  improving  SA  in  Infantry  operations. 

Pleban  et  al.  (2001)  present  additional  results  of  the  soldier-in-the-loop  investigation,  to 
include  decision-making  performance  and  relationships  between  SA  and  decision-making. 

Conclusions 

In  conclusion,  an  exploratory  research  effort  to  examine  the  degree  to  which  SA  could  be 
measured  in  simulated  MOUT  missions  was  conducted.  A  battery  of  objective  and  subjective 
measures  suitable  for  Infantry  operations  was  developed,  based  on  a  detailed  analysis  of  the 
platoon  leader’s  situation  awareness  requirements,  and  tested  The  results  of  this  analysis 
demonstrate  that  these  measures,  particularly  SAGAT,  show  good  promise  for  measuring  SA  in 
future  studies  of  Infantry  operations.  Other  measures  merit  more  testing  and  evolution. 

It  is  important  to  remember  that  these  data  were  obtained  from  scenarios  run  in  a  virtual 
environment  designed  for  individual  combatants.  The  results  of  this  research  effort  provide 
support  for  the  utility  of  such  scenarios  in  both  research  and  training  exercises.  Future  research 
would  benefit,  however,  from  studies  conducted  in  diverse  locations,  including  actual  field 
studies  during  training  exercises  at  a  training  facility  such  as  the  McKenna  MOUT  site  at  Fort 
Banning.  Although  the  difficulties  of  conducting  research  in  such  an  environment  are  great,  they 
are  not  insurmountable,  and  would  provide  important  information  regarding  elements  of  the  SA 
construct  that  are  impossible  to  investigate  at  this  time  in  the  virtual  environment. 

Future  studies  also  should  continue  to  validate  these  measures  of  SA,  specifically  by 
looking  for  links  between  measures  of  performance  or  decision-making  and  measures  of  SA. 
Because  SA  is  a  multidimensional  construct,  it  is  reasonable  to  assume  that  multidimensional 
performance  and  decision-making  measures  will  provide  the  most  utility  for  comparison  with 
measures  of  SA.  Just  as  an  overall  SAGAT  accuracy  score  fails  to  show  effects  for  experience, 
comparison  of  specific  SA  measures  to  overall  performance  measures  are  likely  to  obscure 
relationships  that  may  exist  between  SA  and  specific  decisions  or  outcomes.  Further  work  is 
needed  to  develop  SA  requirements  for  other  Infantry  positions,  following  the  methods  used 
here,  and  to  develop  SA  measures  to  allow  for  research  on  SA  at  different  echelons.  Further 
research  to  refine  and  validate  these  SA  measures  is  also  needed. 
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Appendix  A:  Detailed  Results  of  the  SA  Requirements  Analysis 


The  chart  shown  on  the  following  pages  is  the  result  of  an  SA  Requirements  Analysis  for 
platoon  leaders  during  a  MOUT  mission.  The  methods  used  to  conduct  this  analysis  and 
additional  description  of  the  results  are  given  on  pages  7  through  1 0  of  the  main  report.  The  first 
page  of  the  chart  shows  the  hierarchical  structure  of  the  analysis,  with  Attack,  Secure  and  Hold 
Terrain  (0.0)  as  the  overall  objective.  Seven  primary  goals,  numbered  1 .0  to  7.0,  are  listed 
beneath  this  objective,  with  secondary  goals  identified  beneath  each  of  the  primary  goals.  These 
secondary  goals  share  the  whole  number  of  the  primary  goal  they  fall  below  (i.e.  items  under 
goal  1.0  would  be  1.1, 1.2,  etc.).  Subsequent  pages  each  list  one  secondary  goal  with  all 
subgoals  commonly  associated  with  the  secondary  goal.  The  final  page  of  the  requirements 
analysis  shows  six  categories  of  SA  needs,  which  SMEs  identified  as  important  in  many 
situations.  These  are  listed  separately  to  avoid  the  redundancy  of  frequent  repetition. 

Each  page  that  lists  a  secondary  goal  (A-3  to  A-23)  shows  the  secondary  goal  at  the  top 
of  the  page,  with  the  subgoals  listed  directly  beneath.  Some  subgoals  are  called  into  play  in 
meeting  more  than  one  secondary  goal.  For  brevity,  subgoals  that  are  listed  under  more  than  one 
secondary  goal  are  only  fully  described  in  one  location  in  the  analysis.  Where  these  subgoals  are 
referred  to  on  other  pages,  the  subgoal  number  where  the  complete  description  can  be  found  is 
shown  in  parentheses  beneath  the  name  of  the  subgoal.  Below  each  subgoal  is  a  box  listing 
questions  that  platoon  leaders  might  ask  themselves  to  assist  in  meeting  the  subgoal.  At  the 
bottom  of  the  page,  beneath  the  questions,  is  a  list  of  information  that  the  platoon  leader  would 
like  to  have  in  order  to  answer  the  questions  and  meet  the  subgoal.  Indentation  of  the  SA 
information  serves  two  functions;  first  as  a  general  indicator  of  level  of  SA,  with  Level  3, 
projection,  information  flush  against  the  left  side  of  the  box.  Level  2,  comprehension, 
information  indented  once,  and  Level  1 ,  perception,  information  indented  twice,  and  second  to 
illustrate  information  that  feeds  into  the  higher  level  information  processing  required  for  the 
subgoal.  Therefore,  a  Level  3  SA  item  could  be  indented  once  or  even  twice  if  it  is  used  to 
develop  another  higher  level  comprehension  or  projection  element  more  specific  to  the  goal  in 
question.  In  addition,  items  that  are  found  in  other  portions  of  the  requirements  analysis  are 
shown  in  bold.  Therefore,  items  listed  more  completely  in  the  categories  on  the  final  page  of  the 
analysis  are  shown  in  bold,  as  are  references  back  to  items  listed  under  a  different  subgoal. 
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Situation  Awareness  Categories  for  Platoon  Leaders  in  MOUT 
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Descriptives 
NEWRANK  =  enlisted 


Descriptive  Statistics^ 


N 

Minimum 

Maximum 

Mean 

Std. 

Deviation 

ACib 

Valid  N  (listwise) 

445 

445 

TSUJT 

- J7W 

3.1481 

a.  NEWRANK  =  enlisted 


NEWRANK  =  NCO 


Descriptive  Statistics^ 


N 

Minimum 

Maximum 

Mean 

Std. 

Deviation 

A(3b 

Valid  N  (listwise) 

T^T" 

193 

^6.00 

49.00 

3(5.55^5 

5.53S4 

a.  NEWRANK  =  NCO 


NEWRANK  =  officer 


Descriptive  Statistics® 


N 

Minimum 

Maximum 

Mean 

Std. 

Deviation 

AOb 

Valid  N  (listwise) 

CD  CD 

2±oo 

40.00 

38  5435 

4.6462 

a.  NEWRANK  =  officer 


Appendix  B:  SAGAT  Instructions  to  Platoon  Leaders 


SAGAT  INSTRUCTIONS  TO  SUBJECTS 


Situation  awareness  (SA)  is  critical  to  directing  and  executing  Infantry  operations.  For 
the  purpose  of  this  test,  situation  awareness  is  formally  defined  as:  the  perception  of  the  elements 
in  the  environment  within  a  volume  of  time  and  space,  the  comprehension  of  their  meaning,  and 
the  projection  of  their  status  in  the  near  future.  This  means  your  perception  of  what  is 
happening  in  the  situation,  including  friendly,  enemy,  neutral,  and  non-combatant  disposition, 
actions  and  intentions,  and  what  that  all  means  to  you  as  a  platoon  leader. 

The  Situation  Awareness  Global  Assessment  Technique  (SAGAT)  has  been  developed  to 
objectively  measure  situation  awareness  (SA)  in  manned  simulations. 

•  During  the  trial,  the  simulation  will  be  frozen  at  randomly  determined  intervals  and 
the  visual  scene  blanked. 

» 

•  You  will  be  asked  about  your  knowledge  of  specific  SA  components,  as  you  perceive 
them,  at  that  point  in  time. 

•  The  questions  will  be  presented  on  a  PC.  The  questions  have  been  created  to  allow  for 
quick  and  easy  data  input  using  the  cursor. 

•  You  will  not  be  allowed  to  talk  to  anyone  other  than  the  Test  Director  when 
completing  the  questions. 

•  The  questions  should  be  answered  as  rapidly  as  possible. 

•  Even  if  you  do  not  know  some  of  the  information  exactly,  you  should  make  your  best 
guess.  There  is  no  penalty  for  guessing.  If  you  really  have  no  idea  at  all  of  the 
answer  to  a  question,  you  may  simply  click  on  the  "done"  box  on  the  screen  to  go  on 
to  the  next  question.  You  are  generally  better  off  making  your  best  guess,  however. 

•  Following  the  SAGAT  session  you  will  resume  the  trial  exactly  where  you  left  off. 
You  will  turn  around  and  when  ready  the  simulation  will  be  resumed. 

The  purpose  of  SAGAT  is  to  evaluate  systems  and  training  concepts,  not  to  evaluate  you 
as  an  individual.  You  may  not  be  able  to  answer  many  of  the  questions  you  will  be  asked.  Don’t 
worry  about  this,  as  the  questions  are  intended  to  assess  ideal  SA.  Some  of  this  information  may 
not  be  available  or  may  not  be  adequately  accurate. 

You  will  have  the  opportunity  to  practice  answering  the  SAGAT  questions  before  testing 
begins.  Please  direct  any  questions  you  may  have  to  the  Test  Director. 
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To  begin  the  program  click  on  the  button  marked  “START  SAGAT’. 


1 


Click  on  the  symbol  next  to  each  element  that  is  currently  alive  and  present  in  the 
battlefield  and  drag  it  to  its  current  location.  (You  may  move  the  symbols  on  the  map  if 
they  are  not  placed  correctly).  Indicate  the  location  of  yourself,  your  commander,  the 
main  squad  locations  (with  their  squad  leader),  weapons  squad  location  and  any  teams 
that  have  become  detached  from  their  main  squads  and  are  located  separately.  In 
addition  indicate  the  location  of  any  known  enemies,  enemy  heavy  weapons  locations, 
other  friendly  units  (outside  of  your  platoon)  and  civilians.  Click  on  the  Done  button 
when  you  are  finished  indicating  the  location  of  all  known  parties. 
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,tL.  Kenna 


ROES 


The  location  of  all  the  elements  you  indicated  in  the  first  question  are  presented  on  the 
left.  Click  on  the  button(s)  that  corresponds  to  the  enemy  units  with  the  weakest 
locations  at  this  time.  You  may  indicate  more  than  one.  Click  on  the  Done  button  when 
you  are  finished. 
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The  location  of  all  the  elements  you  indicated  in  the  first  question  are  presented  on  the 
left.  Click  on  the  button(s)  that  corresponds  to  the  enemy  units  with  the  strongest 
locations  at  this  time.  You  may  indicate  more  than  one.  Click  on  the  Done  button  when 
you  are  finished. 


The  location  of  all  the  elements  you  indicated  in  the  first  question  are  presented  on  the 
left.  Click  on  the  button(s)  that  corresponds  to  the  friendly  units  with  the  weakest 
locations  at  this  time.  You  may  indicate  more  than  one.  Click  on  the  Done  button  when 
you  are  finished. 


The  location  of  all  the  elements  you  indicated  in  the  first  question  are  presented  on  the 
left.  Click  on  the  button(s)  that  corresponds  to  the  friendly  units  with  the  strongest 
locations  at  this  time.  You  may  indicate  more  than  one.  Click  on  the  Done  button  when 
you  are  finished. 


9 


The  locations  of  all  the  elements  you  indicated  in  the  first  question  are  presented  on  the 
left.  Click  on  the  button(s)  that  corresponds  to  the  friendly  units  who  are  currently 
exposed  to  enemy  fire/attack.  You  may  indicate  more  than  one.  Click  on  the  Done 
button  when  finished. 
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Does  .the  enemy^nowithe  location^of  your^platoon? 


16 


Indicate  how  many  casualties  your  platoon  has  suffered  up  to  this  point  by  using  the  pull 
down  menu.  Click  on  the  OK  button  when  you  are  done. 
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Indicate  what  actions  you  expect  the  enemy  troops  in  this  scenario  to  take  in  the  next 
five  minutes.  Click  on  the  OK  button  when  you  are  done. 
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18 


WhaVdo  you  expect  civilians  to  do  in  the  next  five  minutes?; 


(Select  ail  that  apply) 


.Become 

hostile' 

Form.a'^if-' 

.  .'crowd 

Move 

2..  positions 

reotfAteck/ 

Disperse  . 

:^®5gGel  in  the’jf- 

;  Nothing 'Q: 

Indicate  what  actions  you  expect  the  civilians  in  this  scenario  to  take  in  the  next  five 
minutes.  Click  on  the  OK  button  when  you  are  done. 
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Indicate  whether  friendly  or  enemy  troops  (or  neither)  currently  have  the  advantage  by 
clicking  on  the  appropriate  button.  Click  on  the  OK  button  when  you  are  done. 
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Which  friendly  elemente  jare  not  in  cpmmuh 


Indicate  whether  any  of  the  above  friendly  troops  have  lost  communications  with  you.  If 
all  are  in  communication  with  you,  click  on  the  None  button.  Click  on  the  OK  button 
when  you  are  done. 


When  you  see  this  screen,  you  are  finished  answering  the  SAGAT  questions.  You  will 
return  to  the  simulation  trial,  following  input  from  the  test  director. 
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Appendix  C:  SABARS  Instrument 


Situation  Awareness  Behaviorally  Anchored  Rating  Scale  (SABARS) 


Rating  Items 

1  ^  : -Sete ; appropri^e'  alert  J ~ ' 7 " ' ;  . .  . r’ ' 

2.  Solicits  information  from  squad  leaders 

3.  Solic^mft^atipn  frq^^  . 

4.  Solicits  information  from  commanders 

5. ^;' Effeds  (^o^inaitipn  wi^'o^^  ' v  :• 

6.  Communicates  key  information  to  commander 

i^y  infoi^  '7  " 

8.  Communicates  key  information  to  other  platoon  leaders  (*) 
;;;;MpnrtorB'(^pany^net^^;ry^';'F';'"^ 

10.  Assesses  information  received 
11J'  AsfeW'perUnent intelligence  jnfc^^ 

12.  Employs  squads  tactically  to  gather  needed  information 

? Wlpys  g raphjc  orotheFcqntrd  squad  execution  (*)  ‘ 

14.  Communicates  to  squads,  situation  and  commander's  intent 

1 5.  ;  Utiices  a  pt^d^  rep^ing  pirac^ ^  •  'i " '  •  ■  ; 

16.  Identifies  critical  mission  tasks  to  squad  leaders 

17.  '  E*^sures“ayer^e^f  appri^h  jr^cd^  .•.•pvV'r  ’  -  v;^  ', 

1 8.  Locates  self  at  vantage  point  to  observe  main  effort 

19.  Deploys  troops  to  ffia [rite  platc>ori  c^mrriunif^^ 

20.  Uses  assets  to  effectively  assess  environment 

21 .  Performs  a  leader’s  recon  to  assess'terrain  and  situation  (*) 

22.  Identifies  observation  points,  avenues  of  approach,  key  terrain, 
obstacles,  cover  and  concealment 

23.  Msesses  key  finds  and  uiiusuar events  ^ 

24.  Discerns  key/critical  information  from  maps,  records,  and 
supporting  site  information  {*) 

25.  piscems  key/critical  [nforma^^^^  '  ' 

26.  Projects  future  possibilities  and  creates  contingency  plans  (*) 

27.  Gathers  follow  up  informafibri'^  ■  ■  ,  - 

28.  Overall  Situation  Awareness  Rating 

*  Designates  items  not  applicable  in  the  present  simulated  missions 
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Appendix  D:  PSAQ  Instrument 


Post  Trial  Participant  Subjective  Questionnaire 


1 .  Please  circle  the  number  below  that  best  describes  how  hard 
you  were  working  during  this  scenario. 


not  hard  1 


extremely 


Comments: 


Please  circle  the  number  that  best  describes  how  well  you 
performed  during  this  scenario 

Comments: 


extremely  poor  1 


-  extremely 
^  well 


Please  circle  the  number  that  best  describes  how  aware  of  Not  aware  of 
the  evolving  situation  you  were  during  the  scenario.  situation 

Comments: 


Completely 
aware  of 
situation 


Appendix  E:  Biographical  Information  Questionnaire 
Biographical  Information  Questionnaire 

Name _  Unit _  Date. 

Please  fill  in  the  blank  or  mark  or  circle  the  appropriate  response. 

1 .  What  is  your  age?  _ Years 

2.  MOS _ 

3.  Rank _ 

4.  Time  in  service  Years _  Months _ 

5.  What  is  the  source  of  your  commission? 

_ ROTC  _ USMA  _ OCS 

6.  What  is  your  current  (or  most  recent)  duty  position?  _ 

How  long  in  this  position?  _ 

7.  What  Army  training  courses  have  you  completed?  Check  all  that  apply. 

_ OSUT/AIT  _ PLDC  _ BNCOC  _ lOBC 

_ BFV  Leader  Course  _ Airborne  _ Ranger 

_ ^Air  Assault  _ Combat  Life  Saver  Course 

_ Other  (please  specify) _ 

8.  How  susceptible  to  motion  or  car  sickness  do  you  feel  you  are? 


1  2 

3  4  5 

6 

7 

not 

moderately 

highly 

susceptible 

susceptible 

susceptible 

9.  Do  you  have  normal  or  corrected  to  normal  20/20  vision?  _ Yes  _ No 
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10.  Are  you  color  blind?  _ ^Yes  _ No 

1 1 .  Are  you  _ right  handed?  _ left  handed? 

12.  My  level  of  confidence  in  using  computers  is 

1  2  3  4  5 

low  average  high 

1 3.  How  many  hours  per  week  do  you  use  computers?  _ hours  per  week 

14.  How  many  times  in  the  last  year  have  you  experienced  a  virtual  reality  game  or 
entertainment? 

0  1  2  3  4  5  6  7  8  9  10  11  12+ 

15.  How  often  have  you  trained  at  the  McKenna  MOUT  site  (not  including  demos)? 

_ not  since  basic  training  _ 1  -3  times  _ more  than  3  times 

1 6.  Have  you  ever  been  in  a  Virtual  Individual  Combatant  (VIC)  simulator  at  the  Land  Warrior 
Test  Bed  before? 

_ Yes  _ No 

If  YES,  which  one(s)?  (Describe  if  you  cannot  remember  the  name) 


17.  Have  you  had  any  other  experience  with  military  computer  simulations? 
_ ^Yes  _ No 

If  yes,  please  describe  briefly  or  give  the  names  of  the  simulators. 
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Appendix  F:  Assault  Scenario  Flow  Chart 

Scenario  B:  Company  Assault  2d  PLT  Mission  from  Building  L  to  A4 


1 .  Co  A,  1-1 1th  Infantry  is  conducting  a  company  assault  on  the  town  of  Kerma.  The  company 
is  attacking  the  town  from  north  to  south.  The  initial  assault  was  successful.  The  1st  Platoon 
attacked  the  H-series  buildings  and  has  successfully  cleared  BLDGs  J2  and  the  I-series  buildings. 
They  are  currently  in  the  I-series  building  preparing  to  assault  the  E-series  buildings.  Their 
follow-on  objectives  are  BLDGs  N  and  P3.  The  3d  Platoon  successfully  attacked  and  cleared 
BLDGs  PI,  P4,  and  C.  They  are  currently  in  BLDG  C,  preparing  to  attack  BLDG  P2.  Their 
follow-on  objective  is  BLDG  P5. 

2.  The  2d  Platoon,  your  platoon,  is  in  the  center.  Your  platoon  has  successfully  cleared  the  G- 
series  buildings  and  is  currently  located  in  BLDG  L.  You  are  at  100%  strength.  You  are 
preparing  to  assault  BLDGs  A4  then  A3,  A2,  and  A1  respectively.  The  commander  has  directed 
you  to  observe  the  assault  from  a  vantage  point  from  the  2d  floor  of  BLDG  L.  Your  platoon  has 
3  rifle  squads  and  1  weapons  squad.  The  Weapons  Squad  Leader,  SSG  Castro,  is  presently  in  the 
hospital  recovering  from  wounds.  Since  the  Weapons  Squad  has  no  team  leaders,  you  have 
attached  the  Weapons  Squad  to  the  3  line  squads.  There  are  no  other  attachments  to  the  platoon. 

3.  The  enemy  is  expected  to  provide  stiff  resistance  as  he  withdraws  his  forces  to  follow-on 
positions  to  the  southern  part  of  town.  The  enemy  is  estimated  at  platoon  strength.  He  has 
mortars,  but  has  not  yet  employed  them.  He  is  fighting  from  well-fortified,  prepared  positions. 

4.  Ration  cycle  is  C,  C,  T.  MEDEVAC  is  restricted  to  escorted  ground  ambulances. 

Ambulances  are  coordinated  through  the  company  commander.  Injured  civilians  will  receive 
medical  assistance.  Captured  enemy  personnel  or  civilian  detainees  will  be  handled  through 
company  channels. 

5.  Rules  of  Engagement  (ROE).  Since  the  local  civilian  population  is  considered  friendly,  the 
rules  of  engagement  are  very  restrictive.  There  will  be  no  weapon  firing  within  the  town  limits 
except  against  confirmed  enemy  locations.  Targets  must  be  clearly  identified  as  hostile.  Every 
effort  must  be  made  to  avoid  civilian  casualties.  Explosives  cannot  be  used  without  permission 
of  the  company  commander.  Weapons  control  status  is  yellow. 

6.  Your  immediate  task  is  to  plan  the  assault  on  BLDG  A4  and  brief  your  squad  leaders. 

7.  What  are  your  questions? 

8.  The  scenario  will  begin  when  you  occupy  the  vantage  point  in  BLDG  L. 
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SET-UP 

1 .  Enemy  machine  gun  in  NE  corner  of  BLDG  P2  orientated  NW  to  N. 

2.  Need  1  small  hole  blown  In  N  side  of  BLDG  A4  (see  computer  cue). 

3.  Need  second  hole  blown  adjacent  to  1st  hole.  The  combination  of  the  two  will  make  a  hole  large  enough  for 
soldiers  to  enter  the  BLDG  (see  computer  cue). 

4.  Need  machine  gun  audio  2  times  (see  computer  cue  and  scenario  for  effectiveness). 

5.  Need  4  dead  civilians  in  BLDG  A4  w/marked  chemical  containers  located  on  the  floor,  in  adjacent  room  to  the 
(entry  point).  Need  abandoned  enemy  tank  in  or  beside  the  east  side  of  BLDG  A4. 


NOTE:  SQD  numbering  system  does  not  match,  but  rather  the  first 
squad  to  perform  a  particular  task  is  the  1st  SQD,  etc. 


PLT  LDR 
Receives 
OPORD. 


Initial  assaults  went  well. 
PLT  is  currently  in  BLDG 
L  ready  to  assault  BLDG 
A.  PLT  LDR  is  at  vantage 
point  on  2d  floor  of  BLDG 
L.  Scenario  begins  with 
PLT  ordering  1  SQD  to 
conduct  breach  of  BLDG 
A. 


Initial  breach 
element  assaults.  3 
of  4  soldiers  KIA  in 
street  by  MG  in 
BLDG  P2  (3d  PLT 
OBJ).  Last  man 
makes  it  to  SW  side 
of  BLDG  L  wounded. 


SQD  LDR 
calls  and 
makes 
report. 


Fails  or  hesitates  to  act 


Unit  is  AGO  1/11  INF 

1st  PLT  Assaults  from  BLDG  I  to  E 

2d  PLT  Assaults  from  BLDG  L  to  A 

3d  PLT  Assaults  from  BLDG  C  to  P2 

Cannot  use  explosives  w/o  okay  of 

CDR. 

Ea  PLT:  3  rifle  SQDs  &  1  WPNs  SQD 
1st  SQD  LDR  w/PLT  <20  days 
No  other  attachments 
Heavy  enemy  presence 
No  heavy  threat 

Civilian  population  overall  friendly 
CDR  informs  LT  of  vantage  point  on  2d 
floor,  south  side  of  BLDG  L. 


1.  CDR  wants  SITREP. 

2.  CDR  again  asks  for 
SITREP. 

3.  CDR  repeats  order. 


I 


SAGAT  Halt 


CDR  guidance 


COMPUTER  CUE:  Show  4 
soldiers  running  from  L,  out 
into  the  street.  3  die.  1 
moves  SW  corner  of  BLDG  L. 
Play  machine  gun  audio 
simultaneously.  Note 
effectiveness 


1d.  Fails  to  call  1st 
and  3d  PLTs  to 
coordinate  for 
suppression. 


DOCTRINAL  ACTIONS 

1.  Reports  to  CDR. 

2.  Request  smoke  mission. 

3.  Coordinate  w/lst  and  3d  PLTs  for 
suppression 

4.  Obtain  status  of  WlA 


If.  Successfully 
completes  task  not  | 
anticipated. 


SAGAT  Halt 


la.  PLT 
LDR  fails  to 
report  to 
CDR 


1b.  Requests 
smoke  mission 
from  CDR. 


1c.  Attempts  to 
call  1st  and  3d 
PLTs  to 
coordinate  for 


CDR  requests 
STREP.  Points 
out  that  this 
situation 
requires  strict 
reporting 
procedures. 

_ 1 _ 

CDR  states 
no  mortars  or 
smoke 
grenades 
available  at 
this  time. 

1  suppression.  | 

CDR  states 
commo  is  major 
problem.  He  will 
relay  messages 
between  PLTs. 

Wait  out. 

CDR  calls.  1st  &  3d  PLTs 
will  suppress  when  2d  PLT 
begins  its  firing. 


Contacts  PSG 
or  SQD  LDR  to 
get  WlA  status. 


Fails  to  get 
WIA  status. 


CDR 

guidance 


PSG  reports 
WIA  slightly 
wounded. 
MEDEVAC  not 
required. 


PSG  reports 
WIA  gut  shot. 
Doesn’t  think 
WIA  will  make  it 
Requests 
MEDEVAC 


Calls  CDR  for 
MEDEVAC 


1g.  Completes  all  tasks. 


F--2 


1 


Continued  on 
next  page 


Continued 

from 

previous  page 


SAGAT  Halt 


CDR  guidance 


1. CDR  asks  for  SITREP. 

2.  CDR  again  asks  for  SITREP, 

3.  CDR  repeats  order. 


Fails  or  hesitates 
to  act 


DOCTRINAL  ACTIONS 

1.  PLT  LDR  issues  FRAGO  for  2d  assault. 

2.  Coordinates  actions  of  other  PLTs  with  CDR. 


PLT  ^ 
LDR 
ACTION 

2  / 


Takes  Action 


AAR  NOTE:  Minimum  FRAGO  should  include  identification  of  breach  TM.  PLT  (-)  provides 
suppression  fires,  order  of  movement,  WPNS  SQD  instructed  to  fire  Javelin.  Backup  plan. 
Coordinate  w/CDR  for  actions  of  other  PLT. 


2a.  Fails  to  coordinate 
w/CDR  for  actions  of 
other  PLTs. 


I 


CDR  asks  for  SITREP. 
Reminds  LT  to  call  when 
ready  so  he  can  coordinate 
1st  and  3d  PLT  fires 


2b.  FRAGO  does  not 
discuss  ID  of  breach  TM, 

PLT  (-)  provides  suppression 
fires,  order  of  movement, 
WPNS  SQD  instructed  to  fire 
Javelin. 


PSG  reminds  LT  to  discuss 
omitted  portion. 


PLT  begins  2d  assault 


2c.  Successfully 
completes  task  not 
anticipated. 


2d. 

Completes  all 
tasks. 


COMPUTER  CUE:  Leave  dead  bodies  in  place. 
Create  two  loud  explosions  to  simulate  Javelin 
firing  &  impact.  Show  breach  TM  (4  personnel) 
moving  to  assault  BLDG  A4.  Play  machine  gun 
audio.  Javelin  hole  not  large  enough  for  manned 
entry  in  exterior  wall.  Entry  fails,  breach  TM 
survives  but  moves  to  road  at  SW  corner  of 
BLDG  L  between  BLDGs  L  and  J2. 


Assault  fails  to  gain  an 
entry.  Breach  force 
survives,  but  is  hiding  at 
corner  of  BLDG  L. 


In  event  PLT  LDR  fails  to  see  assault,  PSG  reports  the  assault  failed  to 
gain  an  entry.  Breach  force  has  survived,  but  is  hiding  in  the  alley 
between  BLDGs  L  &  J2.  SQD  LDR  is  with  breach  TM.  Has  commo 
with  PLT. 


i 

i 


CDR  asks  for  SITREP.  States  1st  PLT  LDR  reports  lost 
all  of  his  breach  force.  Orders  2d  PLT  to  assault  again. 
Use  another  Javelin  to  create  breach.  3d  PLT  will 
support  by  fire. 


PLT  LDR  issues  FRAGO  about  another  assault. 
Coordinates  actions  of  other  PLTs  with  CDR. 


1st  SQD  LDR  reports  PVT  Wampler  is  threatening 
to  shoot  him.  Asks  for  advice. 


Continued  on 
next  page 


Continued  from  previous  page  Fails  or  hesitates  to  act 


1st  SQD  LDR  reports 
PVT  Wampler  is 
threatening  to  shoot  him. 
Asks  for  advice. 


^  PLT  \ 
LDR 
ACTION 
V  3  / 


Takes  Action 


1st  SQD  LDR  reports  PVT 
Wampler  committed  suicide. 


DOCTRINAL  ACTIONS 

1 .  Have  SQD  LDR  take  PVT  Wampler’s  weapon 
away  from  him. 

2.  Restrain  him. 

3.  Post  guard  until  he  can  be  MEDEVACed. 


PLT  LDR  calls  CDR.  Requests  MEDEVAC  for  PVT  Wampler,  If  alive. 


PLT  LDR  executes  FRAGO  for  3d  assault.  Coordinates  w/CDR  for  actions  of  other  PLTs. 


Fails  or 
hesitates 
to  act 


^  PLT  \ 
LDR 
ACTION 
V  4  / 


Takes  Action 


CDR  asks  for  SITREP. 
Reminds  LT  time  is  short, 
need  to  issue  his  FRAGO 
and  execute. 


SAGAT  Halt 


Sends  FRAGO 


DOCTRINAL  ACTIONS 

Minimum  FRAGO  should  include: 

1 .  Identify  breach  TM,  ASLT  TM  (if  used),  and  SPT  TM. 

2.  Fire  control  measures. 

3.  Alternate  plan. 

3.  Order  of  movement/cue  for  movement. 

4.  WPNS  SQD  instructed  to  fire  2  Javelins. 

5.  Coordinate  w/CDR  for  actions  of  other  PLTs. 


4a.  Fails  to  coord. 
w/CDR  for  other 
PLTs. 


CDR  requests 
SITREP.  Requires  LT 
to  hold  until  provided 
execution  time. 


4b.  Fails  to  coord. 
w/CDR  for  actions  of 
other  PLTs.  Prep. 
Javelin  fires. 


CDR  calls  prior  to 
detonation.  Requires  LT 
to  hold  fire  until  provided 
time  line  to  fire. 


4c.  Requests 
permission  to 
blow  hole  in 
BLDG  A4  with 
Javelin. 


CDR  sets 
time  for 
detonation 


4d.  Fails  to 
identify  a 
doctrinal 
action 


PSG  reminds 
LT  to  discuss 


4e. 

Successfully 
completes 
action  not 
anticipated. 


4f.  Com¬ 
pletes  all 
tasks. 


PLT  begins  3d  assault 


COMPUTER  CUE:  Leave  dead  bodies  In  place  from  earlier  attempt.  Keep  2d  breach  TM  next  to 
BLDG  L.  Create  two  sets  loud  explosions  close  together  to  simulate  Javelins  firing  &  impact. 
Show  breach  TM  (4)  moving  to  assault  BLDG  A4.  Play  machine  gun  audio.  Retain  original 
Javelin  Holes  but  increase  size  for  manned  entry  in  exterior  wall.  Breach  TM  gains  entry  into 
building  through  breach  holes.  Have  4  dead  civilians  in  adjacent  room  to  entry  point  in  BLDG  A4. 
Have  2  leaking  chemical  agent  containers  in  same  room  as  dead  civilians  in  BLDG  A4.  Have  1 
abandoned  enemy  tank  in  or  beside  the  E  side  of  BLDG  A4.  Show  remaining  SQD  members 
entering  building  after  breaching  team  to  include  2d  breaching  team  from  SW  corner  of  BLDG  L. 


Breach  TM  gains  entry.  Remaining  SQD  members  follow  breach  element  Into  building. 


Tasked  SQD  LDR  reports  entry.  Has  found  2  large  containers 
marked  with  chemical  agent  signs.  Containers  appear  to  be 
leaking.  There  are  4  dead  civilians  in  the  building  and  an 
abandoned  enemy  tank  on  the  north  side  of  the  building. 
Ordered  SQD  to  mask.  SQD  LDR  asks  for  instructions. 


Continued 


next  page 


F-4 


Continued  from  previous  page 


'  PLT  \ 
LDR 
ACTION 

5  / 


Fails  or  +1  minute  tasked  SQD  dies.  SQD  LDR 

—  hesitates — ►  reports  entire  SQD  dead  and  he  is  seriously  BNDEX 
to  act  ill-  Begin  radio  silence. 

.  \  DOCTRINAL  ACTIONS 

?  1 .  PLT  LDR  has  PLT  immediately  mask. 

/  2.  PLT  LDR  has  SQD  secure  the  area. 

/  3.  Post  guards  to  keep  others  out  of  area. 

Takes  Action  4.  Minimize  exposure  of  friendlies.  Search  tank  to  ensure 

^  not  manned.  Don't  touch  dead  civilians. 

5.  PLT  LDR  submits  SITREP  to  CDR. 


Appendix  G:  Defend  Scenario  Flow  Chart 

Scenario  C:  Defend  Town,  2d  PLT  Mission 

1 .  Co  A,  1-1 1th  Infantry  is  in  the  first  day  of  occupying  the  town  of  Kenna.  The  company  has 
the  mission  to  defend  the  town.  The  enemy  presence  is  considered  light,  capable  of  conducting 
military  operations  in  the  immediate  region  with  forces  less  than  company-size,  supported  by 
mortars.  There  is  no  heavy  threat.  Last  reported  enemy  activity  in  the  region  was  a  platoon¬ 
sized  raid  conducted  in  the  neighboring  town  of  Polo,  3  kilometers  to  the  northwest,  2  nights 
ago.  The  town  population  of  Kenna  is  considered  to  be  friendly.  However,  there  may  be 
insurgents  and  insurgent  sympathizers  within  the  town's  population. 

2.  The  company  has  established  a  defensive  perimeter  with  1st  Platoon  defending  in  the 
southeast  quadrant  of  the  town  from  BLDGs  P5,  P2,  and  A1 .  Your  platoon,  the  2d  Platoon, 
defends  the  southwest  quadrant  of  the  town  from  BLDGs  la-Id,  Ea,  and  P3.  3d  PLT  defends  the 
northwest  quadrant  from  BLDGs  H,  G,  and  Jl.  Company  A  Mortars  defends  the  northeast 
quadrant  from  BLDGs  PI  and  P4.  The  company  CP  is  located  in  the  north  side  BLDG  A4. 

3.  Your  platoon  has  3  rifle  squads  and  1  weapons  squad.  The  Weapons  Squad  Leader,  SSG 
Castro,  is  presently  in  the  hospital  recovering  from  wounds.  Since  the  Weapons  Squad  has  no 
team  leaders,  you  have  attached  the  Weapons  Squad  to  the  3d  Squad  to  take  advantage  of  the 
height  of  Building  Id.  There  are  no  attachments  to  the  platoon. 

4.  The  2d  Platoon  has  been  in  position  for  6  hours.  The  squads  are  positioned  as  shown  in  the 
accompanying  graphic.  The  company  commander  selected  the  north  side  of  BLDG  P3  for  the  2d 
Platoon  CP.  He  has  directed  that  all  platoon  leaders  remain  in  the  general  vicinity  of  their 
respective  CPs  for  communications  purposes.  He  has  directed  that  the  platoons  use  their  platoon 
sergeants  as  much  as  possible  to  realign  positions,  if  necessary. 

5.  Ration  cycle  is  C,  C,  A.  MEDEVAC  is  restricted  to  escorted  ground  ambulances. 

Ambulances  are  coordinated  through  the  company  commander.  Injured  civilians  will  receive 
medical  assistance.  Captured  enemy  personnel  or  civilian  detainees  will  be  handled  through 
company  channels. 

6.  Rules  of  Engagement  (ROE).  Since  the  overall  town  population  is  considered  friendly,  the 
rules  of  engagement  are  very  restrictive.  There  will  be  no  weapon  firing  within  the  town  limits 
except  in  self-defense  or  in  defense  of  the  town  against  a  confirmed  enemy  presence.  Self- 
defense  is  defined  as  a  serious  threat  to  life  or  limb.  A  serious  threat  is  considered  gunfire  or  the 
presence  of  an  uncontrolled  mob  armed  with  life-threatening  weapons  such  as  knives,  pikes, 
metal  poles,  etc.  Every  effort  must  be  made  to  disarm  the  situation  prior  to  the  use  of  deadly 
force.  Weapons  control  status  is  white. 

7.  Your  immediate  task  is  to  brief  your  squad  leaders  as  to  the  current  situation. 

8.  What  are  your  questions? 
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PLT  LDR 
Receives 
OPORD 


Scenario 
starts 
w/PLT  in 
occupied 
positions. 


CDR  tells  PLT 
LDR  that  3d  PLT 
has  reported 
enemy  siting. 
Orders  2d  PLT  to 
send  SQD  to 
reinforce  3d  PLT. 
SQD  to  report  to 
BLDG  J2. 


Falls  or 
hesitates 
to  act"" 


1.  CDR  asks  for  SITREP. 

2.  CDR  again  asks  for 
SITREP. 

3.  CDR  repeats  order. 
SAGAT  Halt 


Unit  Is  Co  A,  1-11thlnf 

1st  PLT  defends  BLDGs  P5.  P2,  A1 

2d  PLT  defends  BLDGs  I,  Ea.  &  P3 

3d  PLT  defends  BLDGs  H,  G.  J1 

Mortar  PLT  defends  BLDGs  PI,  P4 

Ea  PLT;  3  rifle  SQDs&  1  WPNs  SQD 

PLT  CP  mandated  by  CO  CDR  at 

north  side  of  BLDG  P3 

1st  SQD  LDR  w/PLT  <20  days 

No  other  attachments 

Light  enemy  presence 

No  heavy  threat 

Population  friendly 

MEDEVAC  via  ground  mode  thru  CDR 
CO  CP  north  side  BLDG  A4 


Takes_ 

Action 


1.  PLT  LDR  should  not  send 
1st  SQD  LDR- 
Inexperienced. 

2.  To  overcome  commo 
shortage,  PSG  can  be  a  guide. 

3.  If  1st  SQD  is  tasked,  at  +3 
minutes  be  prepared  to  get  lost 
in  1st  PLT  sector  but  report 
you  are  in  position  in  3d  PLT 
sector.  Report  Is  only  sent  if 
PLT  LDR  requires  report  once 
you  are  in  position,  if  not,  radio 
silence  unless  called. 


j - 

CDR  guidance  J 

DOCTRINAL  ACTIONS 

1.  PLT  LDR  sends  FRAGO 
to  tasked  SQD 

2.  Readjusts  positions 
w/remaining  SQDs 

3.  Requires  report  on  link¬ 
up  w/3d  PLT 

4.  Notifies  CO  CDR  of 
movement 


1b.  Tasks  1st  SQD 


SQD  LDR  acknowledges 
FRAGO. 


COMPUTER  CUE; 
Show  movement  of 
1st  SQD  heading  to 
1st  PLT  sector 


At  +3  min,  CDR 
calls  with  report 
that  a  2d  PLT 
SQD  is  lost  In  1st 
PLT  sector. 


la.  Fails  to 
notify  CDR 
of 

movement. 


1c.  Fails  to 
readjust 
remaining 
SQDs 


T 


Id.  Fails  to 
require  link¬ 
up  report. 


CDR  asks 
for  SITREP. 
Gets  report. 
Reminds  LT 
of  reporting 
requirement 


PSG 

recommends 
repositioning 
forces  unless 
3d  SQD  is 
selected. 
WPNS  SQD 
remains 


I 


1e. 

Completes 
action  not 
anticipated. 


PSG 

recom¬ 

mends 

reporting 

link-up. 


If. 

Completes 
all  tasks 
w/2d  or  3d 
SQD 


SQD  LDR 
acknowledges 
FRAGO. 


COMPUTER  CUE; 
Show  movement  of 
SQD  to  BLDG  J2 
lAW  C5. 


+2  minutes  calls.  Reports 
contact  w/3d  PLT  LDR 


Fails  or 
hesitates 
to  act 


,  Takes  ^ 

PLT  LDR 
contacts 

- ^ 

SQD  LDR  reports  that  he  was 
disoriented  but  1st  PLT  LDR 

^  Action^ 

1 

1st  SQD 

W' 

set  him  straight  and  that  he  is 

LDR. 

enroute  to  3d  PLT  sector. 

SAGAT  Halt  W-i  CDR  guidance 


+1  minute  later  SQD  LDR  calls  and 
reports  contact  w/3d  PLT  LDR. 


_ _ ^ 


Continued  on  next  page  <- 


CDR  reports  3d  PLT  siting  2  enemy 
vehicles.  CDR  tells  2d  PLT  LDR  to  send 
1  AT  TM  to  immediately  reinforce  3d  PLT. 


G-2 


Continued  from  previous  page 

CDR  reports  3d  PLT 
siting  2  enemy  / 

vehicles.  CDR  tells 
^  2d  PLT  LDR  to  send 

1  AT  TM  to  reinforce  N 

3d  PLT  ASAP. 


Fails  or  hesitates 
\  to  act 


/  PLT  \ 

LDR  - 

ACTION/^  ‘ 
\  3  / 

\/ Takes  Action 


1. CDR  asks  for  SITREP. 

2.  CDR  again  asks  for 
SITREP. 

3.  CDR  repeats  order. 


SAGAT  Halt 


CDR  guidance 


3a.  Fails  to 
notify  CDR 
of 

movement. 


3b.  Fails 
to  readjust 
remaining 
SQDs 


CDR  asks 
for  SITREP. 
Gets  report. 
Reminds  LT 
about 
reporting. 


3c.  Fails 
to 

require 
report  of 
link-up 
w/3d 
PLT 


PSG  recom¬ 
mends 
repositioning 
forces 


PSG  recom¬ 
mends  reporting 
link-up  using 
detached  SQD 
radio  to  make 
report. 


3f- 

Completes 
all  tasks  and  ' 
sends 
Javelin  TM. 


3e.  Successfully 
completes  action 
not  anticipated. 


PSG  provides 
guidance  on 
sending  ATTM. 
CDR  stated  he 
needs  AT  TM, 
nof  fire  team 


DOCTRINAL  ACTIONS 

1.  Sends  Javelin  TM  from  WPNS 
SQD,  not  rifle  team. 

2.  Readjusts  positions  w/remaining 
SQDs. 

3.  Requires  SQD  report  link-up 
with  3d  PLT. 

4.  Notifies  CDR  of  movement. 


Fails  or  hesitates 
to  act  ^  r? 


SAGAT 

Halt 


/  PLT  \ 
LDR 
ACTION 
\  4  / 


Takes  Action 


Tasked  SQD  reports  departure 
of  ATTM  at  +1  minute. 


Continued 
on  next 
page 


Tasked  SQD  LDR  reports  return  of  fire  team  and 
adjusting  perimeter  at  +3  minutes. 


CDR  reports  1st  PLT  spotted  3  dismounted  OPFOR 
in  wood  line  SW  of  current  company  position. 


COMPUTER  CUE:  Place  5-6 
civilians  milling  around  the 
south  side  of  church,  BLDG  N. 


A  2d  PLT  SQD  LDR  reports  that  a  crowd  of  5  or  6  civilians  is  forming 
on  south  side  of  church  to  his  immediate  rear.  They  appear  hostile. 
Priority  of  call  will  come  from  SQD  LDR  3,  2,  1  in  order. 


FREEZE 

FRAME 


1 .  PSG  recommends  reporting  to 
CDR. 

2.  Finally.  CDR  asks  for  SITREP. 


CDR  guidance  1 


DOCTRINAL  ACTIONS 

1 .  Notifies  CO  CDR  of  situation  and  requests  guidance 

2.  Gives  subordinates  instructions  about  keeping  alert  and 
assigning  someone  to  watch  their  rear 

3.  Warns  troops  to  not  firing  on  potentially  friendly  civilians 

4.  Possibly  requests  reinforcements 


hesitates 

V plt\ 

_ ►/  LDR 

\ ACTION 
\  5  / 
1  ^ Takes  \/ 
Action 


Continued  from  previous  page 


5a.  Fails  to 
notify  CDR. 


CDR  asks 
for  SURER 
on  report  of 
possible  civil 
disturbance. 


5b.  Does  not 
give  SQDs 
instructions 
about  activity 
in  their  rear 
area. 


PSG 

recommends 
warning  soldiers 
to  assign 
someone  to 
watch  civilians. 


5c.  Does  not 
warn  troops  of 
status  of 
civilians. 


Soldier  shoots  civilian  in  retaliation  for 
civilian  throwing  a  rock. 


PSG  reports  he  and  medic  are 
enroute  to  aid  the  injured  civilian. 


5d.  Requests 
reinforce¬ 
ments. 


CDR 

disapproves 

request. 


COMPUTER  CUE:  Two  shots  fired  at  dose  range. 
One  of  civilians  falls  to  the  ground. 


If  LT  does  not  see  event,  a  SOD  LDR  reports  civilian 
shot  for  throwing  rocks.  Priority  of  call  comes  from 
SOD  LDR  1,2,  3. 


Fails  or  ^ 
hesitates  to  act’ 


/  PLT  \ 
LDR 

ACTION  / 

\®/ 

\/  Takes 


1 .  PSG  recommends  reporting  to  CDR. 

2.  CDR  calls  for  cease  fire  and  wants 
shooting  investigated. 


SAGAT  Halt 


I  CDR  guidance 


DOCTRINAL  ACTIONS 

1 .  Calls  for  cease  fire  on  PLT  net. 

2.  Notifies  CO  CDR  of  situation. 

3.  Gives  subordinates  instructions. 

4.  Requests  MEDEVAC. 


6e. 

Successfully 
completes 
task  not 
anticipated. 


6f. 

Completes 
all  tasks. 


Continued 
on  next 
page 


G-4 


Continued  from  previous  page 


Appendix  H:  Data  Analysis  Results 


Table  H-1 .  Anova  Table  for  Enemy  Location 


DF 

Sum  of 

Squares 

Mean 

Square 

F- Value 

_ _ _ _ 

P-Value 

Experience 

1 

3991.059 

3991.059 

KBI 

0.032 

Scenario 

1 

101.264 

101.264 

HIB 

Halt 

2 

949.122 

474.561 

Experience  *  Scenario 

1 

15.962 

15.962 

■ililEl 

Experience  *  Halt 

2 

37.466 

18.733 

0.023 

0.978 

Scenario  *  Halt 

2 

4720.104 

2360.052 

Experience  *  Scenario  *  Halt 

2 

142.150 

71.075 

Residual 

69 

57265.829 

829.940 

Table  H-2.  Table  of  Means  for  Enemy  Location 


Mean 

Experienced,  Assault,  1 

69.347 

Experienced,  Assault,  2 

58.127 

Experienced,  Assault,  3 

60.154 

Experienced,  Defend,  1 

44.880 

Experienced,  Defend,  2 

64.751 

Experienced,  Defend,  3 

73.943 

Novice,  Assault,  1 

54.725 

Novice,  Assault,  2 

49.362 

Novice,  Assault,  3 

44.000 

Novice,  Defend,  1 

33.660 

Novice,  Defend,  2 

46.907 

Novice,  Defend,  3 

58.127 

Table  H-3.  Anova  Table  for  Own  Platoon  Location 


DF 

Sum  of 

Squares 

Mean 

Square 

F- Value 

P-Value 

Experience 

1 

277.754 

277.754 

5.190 

0.026 

Scenario 

1 

18.390 

18.390 

0.344 

0.560 

Halt 

2 

220.808 

110.404 

IQjjgg 

Experience  *  Scenario 

1 

156.226 

156.226 

Experience  *  Halt 

2 

117.075 

IQQgl 

mgn 

Scenario  *  Halt 

2 

73.749 

Experience  *  Scenario  *  Halt 

2 

224.197 

112.098 

2.095 

0.131 

Residual 

69 

3692.499 

53.514 

H-1 


Table  H-4.  Table  of  Means  for  Own  Platoon  Location 


Mean 

Experienced,  Assault,  1 

75.378 

Experienced,  Assault,  2 

68.717 

Experienced,  Assault,  3 

Experienced,  Defend,  2 

74.846 

Experienced,  Defend,  3 

69.993 

Novice,  Assault,  1 

74.851 

Novice,  Assault,  2 

72.613 

Novice,  Assault,  3 

67.644 

Novice,  Defend,  1 

66.404 

Novice,  Defend,  2 

69.922 

Novice,  Defend,  3 

67.566 

Table  H-5.  Anova  Table  for  Friendly  Location 


m 

Sum  of 

Squares 

Mean 

Square 

m 

Experience 

1 

732.506 

732.506 

1.103 

0.297 

Scenario 

1 

4737.692 

4737.692 

0.009 

Halt 

2 

400.877 

200.439 

0.740 

Experience  *  Scenario 

1 

2127.708 

2127.708 

3.204 

2 

157.374 

0.118 

Scenario  *  Halt 

2 

1021.989 

510.995 

0.769 

0.467 

Experience  *  Scenario  *  Halt 

2 

472.124 

236.062 

0.355 

0.702 

Residual 

69 

45823.548 

664.109 

Table  H-6.  Table  of  Means  for  Friendly  Location 


Mean 

Experienced,  Assault,  1 

40.028 

Experienced,  Assault,  2 

56.834 

Experienced,  Assault,  3 

44.620 

Experienced,  Defend,  1 

40.358 

Experienced,  Defend,  2 

35.321 

Experienced,  Defend,  3 

50.632 

Novice,  Assault,  1 

63.075 

Novice,  Assault,  2 

63.075 

Novice,  Assault,  3 

64.235 

Novice,  Defend,  1 

36.548 

Novice,  Defend,  2 

34.690 

Novice,  Defend,  3 

42.338 

H-2 


Table  H-7.  Anova  Table  for  Total  Elements  Located 


DF 

Sum  of 

Squares 

Mean 

Square 

F-Value 

P-Value 

Experience 

1 

66.796 

66.796 

0.440 

0.509 

Scenario 

1 

2299.593 

2299.593 

15.151 

<0.001 

Halt 

2 

41.967 

20.983 

0.871 

Experience  *  Scenario 

1 

241.930 

241.930 

bkei 

0.211 

Experience  *  Halt 

2 

18.225 

9.112 

0.942 

Scenario  *  Halt 

2 

229.871 

0.473 

Experience  *  Scenario  *  Halt 

2 

56.010 

28.005 

0.185 

0.832 

Residual 

69 

10472.702 

151.778 

Table  H-8.  Table  of  Means  for  Total  Elements  Located 


Mean 

Experienced,  Assault,  1 

65.870 

Experienced,  Assault,  2 

63.965 

Experienced,  Assault,  3 

63.965 

Experienced,  Defend,  1 

56.427 

Experienced,  Defend,  2 

57.036 

Experienced,  Defend,  3 

58.688 

Novice,  Assault,  1 

68.704 

Novice,  Assault,  2 

65.094 

Novice,  Assault,  3 

64.933 

Novice,  Defend,  1 

48.158 

Novice,  Defend,  2 

51.428 

Novice,  Defend,  3 

56.711 

H-3 


Table  H-9.  Anova  Table  for  Highest  Threat 


DF 

Sum  of  Squares 

Mean 

Square 

F- Value 

P- Value 

Experience 

1 

12200.906 

12200.906 

9.362 

0.003 

Scenario 

1 

3008.755 

3008.755 

2.309 

0.133 

Halt 

2 

371.754 

185.877 

0.143 

0.867 

Experience  *  Scenario 

1 

23.108 

23.108 

0.018 

Experience  *  Halt 

2 

2076.151 

1038.075 

0.797 

Scenario  *  Halt 

2 

3095.797 

1547.899 

1.188 

0.311 

Experience  *  Scenario  *  Halt 

2 

2751.750 

1375.875 

1.056 

0.354 

Residual 

68 

88617.560 

1303.199 

Table  H-10.  Table  of  Means  for  Highest  Threat 


Mean 

Experience,  Assault,  1 

Experience,  Assault,  2 

■riRiaiil 

Experience,  Assault,  3 

Experience,  Defend,  1 

44.880 

Experience,  Defend,  2 

Experience,  Defend,  3 

Novice,  Assault,  1 

■aaggii 

Novice,  Assault,  2 

39.270 

Novice,  Assault,  3 

52.360 

Novice,  Defend,  1 

■tMEEW 

Novice,  Defend,  2 

Novice,  Defend,  3 

mtsssm 

Table  H-1 1.  Anova  Table  for  Weakest  Enemy 


DF 

Sum  of  Squares 

Mean 

Square 

F-Value 

iQyi 

Experience 

1 

440.816 

440.816 

0.347 

Scenario 

1 

14823.240 

14823.240 

11.666 

0.001 

Halt 

2 

3947.675 

1973.838 

1.553 

0.219 

Experience  *  Scenario 

1 

17.666 

17.666 

■WIIEl 

0.907 

Experience  *  Halt 

2 

1691.432 

845.716 

m&tki 

Scenario  *  Halt 

2 

6767.729 

3383.864 

2.663 

Experience  *  Scenario  *  Halt 

2 

2256.577 

1128.288 

0.888 

0.416 

Residual 

69 

87675.650 

1270.662 

H-4 


Table  H-12.  Table  of  Means  for  Weakest  Enemy 


Mean 

Experience,  Assault,  1 

11.220 

Experience,  Assault,  2 

22.440 

Experience,  Assault,  3 

Experience,  Defend,  1 

67.320 

Experience,  Defend,  2 

33.660 

Experience,  Defend,  3 

33.660 

Novice,  Assault,  1 

13.090 

Novice,  Assault,  2 

Novice,  Assault,  3 

■natiEB 

Novice,  Defend,  1 

■aaniii] 

Novice,  Defend,  2 

11.220 

Novice,  Defend,  3 

56.100 

Table  H-13.  Anova  Table  for  Strongest  Enemy 


DF 

Experience 

1 

HUSIISI 

Scenario 

1 

70.414 

Halt 

2 

1139.829 

IfHOOg 

Experience  *  Scenario 

1 

3454.945 

Experience  *  Halt 

2 

857.340 

428.670 

■iiaaa 

Scenario  *  Halt 

2 

16368.887 

6.430 

0.003 

Experience  *  Scenario  *  Halt 

2 

857.340 

428.670 

0.337 

0.715 

Residual 

69 

87822.519 

1272.790 

Table  H-14.  Table  of  Means  for  Strongest  Enemy 


Mean 

Experience,  Assault,  1 

78.540 

Experience,  Assault,  2 

78.540 

Experience,  Assault,  3 

33.660 

Experience,  Defend,  1 

33.660 

Experience,  Defend,  2 

44.880 

Experience,  Defend,  3 

67.320 

Novice,  Assault,  1 

26.180 

Novice,  Assault,  2 

39.270 

Novice,  Assault,  3 

13.083 

Novice,  Defend,  1 

22.440 

Novice,  Defend,  2 

33.660 

Novice,  Defend,  3 

56.100 

H-5 


Table  H-15.  Anova  Table  for  Weakest  Friendly 


DF 

Sum  of  Squares 

Mean 

Square 

F-Value 

P- Value 

Experience 

1 

580.283 

580.283 

0.381 

0.539 

Scenario 

1 

4053.395 

4053.395 

2.662 

0.108 

Halt 

2 

622.439 

311.220 

0.204 

0.816 

Experience  *  Scenario 

1641.337 

1641.337 

1.078 

0.303 

Experience  *  Halt 

2 

920.574 

460.287 

0.302 

Scenario  *  Halt 

2 

7824.712 

3912.356 

2.570 

Experience  *  Scenario  *  Halt 

2 

936.467 

468.233 

0.308 

Residual 

66 

100486.018 

1522.515 

Table  H-16.  Table  of  Means  for  Weakest  Friendly 


Mean 

Experience,  Assault,  1 

Experience,  Assault,  2 

Experience,  Assault,  3 

44.880 

Experience,  Defend,  1 

11.220 

Experience,  Defend,  3 

Novice,  Assault,  1 

62.832 

Novice,  Assault,  2 

52.360 

Novice,  Assault,  3 

52.360 

Novice,  Defend,  1 

■aaEEfi] 

Novice,  Defend,  2 

KB 

Novice,  Defend,  3 

Table  H-17.  Anova  Table  for  Strongest  Friendly 


DF 

Sum  of  Squares 

Mean 

Square 

Experience 

1 

4537.133 

4537.133 

6.504 

0.013 

Scenario 

1 

8812.172 

8812.172 

12.633 

0.001 

Halt 

2 

557.852 

0.800 

0.454 

Experience  *  Scenario 

1 

286.450 

286.450 

0.411 

0.524 

Experience  *  Halt 

2 

536.748 

268.374 

0.682 

Scenario  *  Halt 

2 

1861.895 

930.948 

0.270 

Experience  *  Scenario  *  Halt 

2 

1489.227 

744.613 

■Bisg 

0.350 

Residual 

65 

45342.033 

U-6 


Table  H-18.  Table  of  Means  for  Strongest  Friendly 


Mean 

Experience,  Assault,  1 

54.725 

Experience,  Assault,  2 

28.545 

Experience,  Assault,  3 

65.450 

Experience,  Defend,  1 

33.118 

Experience,  Defend,  2 

35.678 

Experience,  Defend,  3 

27.024 

Novice,  Assault,  1 

73.177 

Novice,  Assault,  2 

65.670 

Novice,  Assault,  3 

67.815 

Novice,  Defend,  1 

45.870 

Novice,  Defend,  2 

42.311 

Novice,  Defend,  3 

42.311 

Table  H-19.  Anova  Table  for  Exposed  Friendly 


DF 

Sum  of  Squares 

Mean 

Square 

F- Value 

P-Value 

Experience 

1 

1022.286 

1022.286 

0.817 

0.369 

Scenario 

1 

28293.562 

28293.562 

22.625 

Halt 

1012.748 

506.374 

0.405 

0.669 

Experience  *  Scenario 

1 

233.831 

233.831 

0.187 

0.667 

Experience  *  Halt 

2 

349.892 

174.946 

0.140 

Scenario  *  Halt 

2 

5634.433 

2817.217 

2.253 

■Boa 

Experience  *  Scenario  *  Halt 

2 

1012.748 

506.374 

Residual 

68 

85037.216 

1250.547 

IBIH 

Table  H-20.  Table  of  Means  for  Exposed  Friendly 


Mean 

Experience,  Assault,  1 

11.220 

Experience,  Assault,  2 

0.000 

Experience,  Assault,  3 

22.440 

Experience,  Defend,  1 

67.320 

Experience,  Defend,  2 

56.100 

Experience,  Defend,  3 

33.660 

Novice,  Assault,  1 

26.180 

Novice,  Assault,  2 

13.090 

Novice,  Assault,  3 

26.180 

Novice,  Defend,  1 

56.100 

Novice,  Defend,  2 

67.320 

Novice,  Defend,  3 

44.880 

H-7 


Table  H-2 1 .  Anova  Table  for  My  Location  Known 


jsm 

Sum  of  Squares 

Mean 

Square 

m 

Experience 

1 

634.475 

634.475 

1.553 

0.217 

Scenario 

1 

3454.362 

3454.362 

8.453 

0.005 

Halt 

2 

2678.893 

1339.446 

3.278 

0.044 

Experience  *  Scenario 

1 

634.475 

634.475 

1.553 

0.217 

Experience  *  Halt 

2 

422.983 

211.492 

■iMia 

Scenario  *  Halt 

2 

2678.893 

1339.446 

Experience  *  Scenario  *  Halt 

2 

422.983 

211.492 

0.518 

0.598 

Residual 

69 

28198.870 

408.679 

Table  H-22.  Table  of  Means  for  My  Location  Known 


ksh 

Experience,  Assault,  1 

Experience,  Assault,  2 

Experience,  Assault,  3 

MK!il 

Experience,  Defend,  1 

M3g!i! 

Experience,  Defend,  2 

78.540 

Experience,  Defend,  3 

Novice,  Assault,  1 

Novice,  Assault,  2 

IlflggQ 

Novice,  Assault,  3 

78.540 

Novice,  Defend,  2 

Novice,  Defend,  3 

■ussy!] 

Table  H-23 .  Anova  Table  for  Number  of  Casualties 


f 

Sum  of  Squares 

Mean 

Square 

n 

Experience 

1 

158.619 

158.619 

0.197 

0.659 

Scenario 

1 

26806.551 

26806.551 

33.229 

Halt 

2 

505.230 

252.615 

0.313 

Experience  *  Scenario 

1 

158.619 

158.619 

0.197 

0.659 

Experience  *  Halt 

2 

1492.190 

746.095 

0.925 

0.401 

Scenario  *  Halt 

2 

505.230 

252.615 

0.313 

Experience  *  Scenario  *  Halt 

2 

1492.190 

746.095 

0.925 

HE&O 

Residual 

69 

55663.394 

806.716 

H-8 


Table  H-24.  Table  of  Means  for  Number  of  Casualties 


Mean 

Experience,  Assault,  1 

44.880 

Experience,  Assault,  2 

33.660 

Experience,  Assault,  3 

56.100 

Experience,  Defend,  1 

78.540 

Experience,  Defend,  2 

78.540 

Experience,  Defend,  3 

78.540 

Novice,  Assault,  1 

26.180 

Novice,  Assault,  2 

52.360 

Novice,  Assault,  3 

39.270 

Novice,  Defend,  1 

78.540 

Novice,  Defend,  2 

78.540 

Novice,  Defend,  3 

78.540 

Table  H-25.  Anova  Table  for  Next  Enemy  Action 


DF 

Sum  of  Squares 

Mean 

Square 

F-Value 

P-Value 

Experience 

1 

7.094 

7.094 

0.009 

Scenario 

1 

25965.910 

25965.910 

33.217 

Halt 

2 

21189.457 

10594.729 

13.553 

■Hlllllil 

Experience  *  Scenario 

1 

1120.920 

1120.920 

1.434 

Experience  *  Halt 

2 

1741.582 

870.791 

1.114 

Scenario  *  Halt 

2 

12480.671 

6240.335 

7.983 

Experience  *  Scenario  *  Halt 

2 

1463.845 

731.923 

0.936 

0.397 

Residual 

68 

53155.398 

781.697 

Table  H-26.  Table  of  Means  for  Next  Enemy  Action 


Mean 

Experience,  Assault,  1 

78.540 

Experience,  Assault,  2 

78.540 

Experience,  Assault,  3 

11.220 

Experience,  Defend,  1 

26.180 

Experience,  Defend,  2 

11.220 

Experience,  Defend,  3 

0.000 

Novice,  Assault,  1 

78.540 

Novice,  Assault,  2 

52.360 

Novice,  Assault,  3 

13.090 

Novice,  Defend,  1 

17.844 

Novice,  Defend,  2 

17.844 

Novice,  Defend,  3 

22.440 
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Table  H-27.  Anova  Table  for  Next  Civilian  Action 


!■ 

Sum  of 

Squares 

Mean 

Square 

F- Value 

P- Value 

Experience 

1 

4143.667 

4143.667 

HEEB 

0.085 

Scenario 

1 

1323.780 

1323.780 

0.327 

Halt 

2 

13022.395 

6511.197 

4.795 

0.011 

Experience  *  Scenario 

1 

195.825 

195.825 

0.144 

0.705 

Experience  *  Halt 

2 

426.900 

213.450 

0.157 

0.855 

Scenario  *  Halt 

2 

10578.493 

5289.246 

3.895 

0.025 

Experience  *  Scenario  *  Halt 

2 

426.900 

213.450 

0.157 

0.855 

Residual 

69 

93702.494 

1358.007 

Table  H-28.  Table  of  Means  for  Next  Civilian  Action 


mm 

Experience,  Assault,  1 

56.100 

Experience,  Assault,  2 

56.100 

Experience,  Assault,  3 

44.880 

Experience,  Defend,  1 

78.540 

Experience,  Defend,  2 

11.220 

Experience,  Defend,  3 

33.660 

Novice,  Assault,  1 

39.270 

Novice,  Assault,  2 

39.270 

Novice,  Assault,  3 

26.180 

Novice,  Defend,  1 

56.100 

Novice,  Defend,  2 

11.220 

Novice,  Defend,  3 

Table  H-29.  Anova  Table  for  Advantage 


fsm 

Sum  of  Squares 

Mean 

Square 

F-Value 

n 

Experience 

1 

565.936 

565.936 

0.417 

Scenario 

llllllin 

565.936 

565.936 

0.417 

Halt 

2 

3399.530 

Experience  *  Scenario 

1 

3620.813 

3620.813 

■acwii 

Experience  *  Halt 

2 

203.659 

101.829 

BBS 

Scenario  *  Halt 

2 

2177.579 

1088.790 

0.803 

Experience  *  Scenario  *  Halt 

2 

109.662 

54.831 

0.040 

Residual 

69 

93555.625 

1355.879 
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Table  H-30.  Table  of  Means  for  Advantage 


Mean 

Experience,  Assault,  1 

56.100 

Experience,  Assault,  2 

56.100 

Experience,  Assault,  3 

Experience,  Defend,  1 

Experience,  Defend,  2 

Experience,  Defend,  3 

Novice,  Assault,  1 

65.450 

Novice,  Assault,  2 

65.450 

Novice,  Assault,  3 

39.270 

Novice,  Defend,  1 

56.100 

Novice,  Defend,  2 

Novice,  Defend,  3 

44.880 

Table  H-3 1 .  Anova  Table  for  Not  in  Communication 


DF 

Sum  of  Squares 

Mean 

Square 

F-Value 

P-Value 

Experience 

1 

1810.630 

1810.630 

1.563 

0.215 

Scenario 

1 

103.115 

103.115 

0.089 

0.766 

Halt 

2 

1488.758 

744.379 

0.529 

Experience  *  Scenario 

1 

6954.292 

6954.292 

MMia 

0.017 

Experience  *  Halt 

2 

132.434 

66.217 

0.057 

0.945 

Scenario  *  Halt 

2 

1488.758 

744.379 

0.643 

0.529 

Experience  *  Scenario  *  Halt 

2 

132.434 

66.217 

0.057 

0.945 

Residual 

69 

79908.919 

1158.100 

Table  H-32.  Table  of  Means  for  Not  in  Communication 


Mean 

Experience,  Assault,  1 

11.220 

Experience,  Assault,  2 

11.220 

Experience,  Assault,  3 

11.220 

Experience,  Defend,  1 

29.063 

Experience,  Defend,  2 

13.247 

Experience,  Defend,  3 

40.283 

Novice,  Assault,  1 

39.270 

Novice,  Assault,  2 

39.270 

Novice,  Assault,  3 

39.270 

Novice,  Defend,  1 

17.844 

Novice,  Defend,  2 

11.216 

Novice,  Defend,  3 

26.247 
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Table  H-33.  SABARS  Factor  Analysis  -  Factor  Loadings 


Factor  1- 

Gathering 

Information 

Factor  2-Focusing 
Out  Vs  In 

Factor  3-Seeking 
Key  Information 

Factor  4-Focusing 
on  Big  Picture 

Alert  Level 

-0.037 

-0.072 

0.160 

SL  Info 

-0.778 

0.020 

0.250 

CO  Info 

-0.178 

0.274 

0.588 

Commo  to  CO 

0.185 

0.304 

Commo  to  SL 

Monitors  net 

0.208 

Assess  Info 

0.173 

0.388 

0.267 

Intel  requested 

0.256 

0.337 

Intel  gathered 

-0.027 

-0.129 

-0.118 

Give  Cl  &  big  pic 

0.131 

-0.167 

-0.027 

■  ^  0.6^^ 

Uses  SRP 

0.825 

0.187 

-0.197 

0.264 

Ids  Critical  tasks 

-0.038 

i 

C  -0.817 

0.011 

0.077 

Cover  Approach 

-0.170 

-0.105 

%  0.629 

0.215 

Vantage  Pt 

0.117 

0.183 

0.629 

0.062 

Asset  recon 

.  0.862 

0.169 

-0.004 

OCOKA 

0.695 

-0.037 

-0.264 

Key  finds 

v--':0.339 

:-'"':^;0.405 

:  ^  v-^  0.603 

0.205 

Key  info-reports 

0.246 

-  ^0.403 

0.175 

Follow  up  info 

0.611 

0.605 

0.051 

<.001 

Overall  SA 

0.547 

0.558 

-0.009 

*  all  items  with  loadings  >  .30  included  in  combined factor  score 


Table  H-34.  SABARS  Factor  Analysis  -Eigenvalues  and  percent  of  variance  accounted  for  by 
factors 


Eigenvalues 


Magnitude 

Variance  Prop. 

Factor  1  -  Gathering  Information 

7.113 

0.356 

Factor  2  -  Focusing  Out  Vs.  In 

2.361 

0.118 

Factor  3  -  Seeking  Key  Information 

2.32 

0.116 

Factor  4  -  Focusing  on  Big  Picture 

1.521 

0.076 
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Table  H-35.  Correlation  Matrix  for  all  SA  Measures 


1-Info 

Gather 

2-Focus 
Out  V. 

In 

3-Seek 

Key 

Info 

4-Big 

Picture 

PSAQ 

Workload 

PSAQ 

Performance 

PSAQ 

SA 

Enemy 

ID 

1-Info  Gathering 

1 

0.557 

-0.016 

0.097 

0.339 

0.413 

2-Focus  Out  V.  In 

0.557 

1 

0.368 

0.56 

-0.226 

■graa 

3-Seeks  Key  Info 

0.685 

0.368 

1 

0.367 

0.161 

ESS] 

0.269 

0.623 

mm 

1 

-0.085 

0.001 

WKVM 

Self-Rated  Workload 

-0.016 

■nma 

0.045 

1 

-0.375 

-0.143 

-0.287 

Self  -Rated  Performance 

0.097 

-0.226 

0.161 

-0.085 

-0.375 

1 

0.26 

Self  -  Rated  SA 

0.339 

0.106 

0.471 

0.001 

miEi 

0.26 

1 

Mm 

Enemies  Located 

0.413 

0.153 

0.269 

.0.558 

-0.287 

0.167 

■ilE! 

Own  Platoon  Located 

0.102 

0.121 

0.129 

0.141 

0.188 

0.077 

0.37 

Other  Friendlies  Located 

-0.093 

0.183 

-0.036 

0.247 

0.388 

-0.21 

-0.027 

0.263 

Total  Elements  Located 

0.032 

0.262 

-0.038 

0.322 

0.354 

-0.214 

-0.06 

0.436 

Highest  Threat 

0.09 

-0.155 

0.202 

0.146 

BflTiBe! 

0.453 

Weakest  Enemy 

0.028 

0.046 

0.115 

-0.015 

0.21 

0.262 

0.193 

Strongest  Enemy 

0.282 

0.081 

0.299 

0.334 

0.2 

0.166 

-0.04 

Weakest  Friendly 

-0.298 

-0.088 

-0.233 

-0.062 

0.268 

0.017 

gfflllM 

Strongest  Friendly 

-0.455 

-0.078 

-0.464 

-0.2 

0.162 

-0.139 

miEi 

Exposed  Friendly 

0.031 

0.22 

0.052 

-0.258; 

-0.157 

0.205 

gtlgE! 

F  Location  Known 

-0.166 

0.218 

-0.255 

-0.249 

0.139 

-0.132 

El^ 

#  Casualties 

0.371 

0.284 

0.551 

0.219 

0.369 

0.551 

-0.122 

Expected  Enemy  Actions 

-0.124 

-0.057 

-0.349: 

0.02 

-0.145 

-0.37 

-0.119 

Expected  Civilian  Actions 

0.255 

0.314 

-0.04; 

0.18 

-0.184 

-0.125 

-0.004 

0.276 

Advantage 

0.105 

-0.173 

0.08 

0.067 

0.299 

0.23 

0.366 

Not  in  Communication 

-0.175 

-0.07 

-0.225 

0.1 

-0.354 

0.261 

-0.243 

0.268 
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Table  H-35  (continued)  Correlation  Matrix  for  all  SA  Measures 


Own 

Troop 

ID 

Adj. 

Friendly 

ID 

Total 

ID 

Highest 

Threat 

Weakest 

Enemy 

Strongest 

Enemy 

Weakest 

Friendly 

Strongest 

Friendly 

1  -Info  Gathering 

0.102 

-0.093 

0.032 

0.09 

0.028 

0.282 

-0.298 

: -0.455 

2-Focus  Out  V.  In 

0.121 

0.183 

0.262 

-0.155 

■BEES 

3 -Seeks  Key  Info 

0.129 

-0.036 

-0.038 

0.202 

IQII 

■BiSLSj 

mm 

-0.2 

Self-Rated  Workload 

0.188 

0.388 

0.354 

0.039 

-0.239 

0.162 

Self  -Rated  Performance 

0.155 

-0.21 

-0.214 

0.111 

0.21 

0.166 

0.017 

-0.139 

Self  -  Rated  SA 

0.077 

-0.027 

-0.06 

-0.074 

0.262 

-0.11 

Enemies  Located 

0.263 

mm 

0.453 

0.193 

mm 

Own  Platoon  Located 

0.508 

IQEi 

■EEffi 

Other  Friendlies  Located 

0.339 

1 

0.902 

0.15 

-0.147 

0.244 

BsEsEJJ 

Total  Elements  Located 

0.5 

■MiL«Iili8J 

1 

0.295 

-0.057 

0.398 

Highest  Threat 

0.295 

1 

EQEE 

Weakest  Enemy 

-0.147 

-0.057 

1 

Strongest  Enemy 

0.398 

1 

-0.212 

mm 

Weakest  Friendly 

1 

Strongest  Friendly 

bees 

WBSSSB 

BEgj 

HHH! 

Exposed  Friendly 

B»i8wa 

EBiU^ 

mm 

F  Location  Known 

■WPB 

0.119 

■6^1 

IHKE& 

BES! 

IHSES 

#  Casualties 

-0.119 

-0.151 

-0.285 

-0.204 

0.312 

IIES&h 

Expected  Enemy  Actions 

-0.153 

0.299 

0.339 

0.137 

-0.676 

hqh 

Expected  Civilian  Actions 

-0.22 

0.123 

0.217 

-0.063 

una 

HEH 

eesbe 

0.041 

Advantage 

0.123 

-0.186 

-0.155 

0.305 

MlBKH 

■beie 

Not  in  Communication 

-0.059 

0.178 

0.131 

-0.014 

-0.094 

-0.073 

0.152 

hhd 
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Table  H-35  (continued)  Correlation  Matrix  for  all  SA  Measures 


Exposed 

Friendly 

F 

Location 

Known 

§ 

Casualties 

Next 

Enemy 

Actions 

Next 

Civilian 

Actions 

Advantage 

Not  in 

Communication 

1-Info  Gathering 

0.031 

-0.166 

0.371 

-0.124 

0.255 

0.105 

-0.175 

2-Focus  Out  V.  In 

0.22 

0.218 

0.284 

-0.057 

0.314 

-0.173 

-0.07 

3 -Seeks  Key  Info 

0.551 

-0.349 

-0.04 

0.08 

-0.225 

4-Big  Picture  Focus 

-0.258 

-0.249 

0.219 

0.02 

0.18 

0.1 

Self-Rated  Workload 

-0.157 

0.139 

-0.008 

0.314 

-0.184 

-0.354 

Self  -Rated  Performance 

0.205 

-0.132 

0.369 

-0.145 

-0.125 

Self  -  Rated  SA 

0.346 

-0.189 

0.551 

-0.37 

-0.004 

0.23 

-0.243 

Enemies  Located 

-0.122 

-0.119 

0.276 

0.366 

0.268 

Own  Platoon  Located 

-0.276 

-0.119 

-0.153 

0.123 

-0.059 

Other  Friendlies  Located 

-0.289 

-0.151 

0.299 

■ilfcB 

-0.186 

0.178 

Total  Elements  Located 

-0.346 

0.119 

-0.285 

0.339 

-0.155 

0.131 

Highest  Threat 

0.05 

-0.204 

0.137 

-0.063 

0.305 

-0.014 

Weakest  Enemy 

0.277 

0.312 

■iiagg 

0.125 

-0.094 

Strongest  Enemy 

-0.323 

HEIS 

■BS9] 

ihseIb 

0.077 

-0.073 

Weakest  Friendly 

-0.129 

HSS! 

0.293 

bob 

0.017 

Strongest  Friendly 

-0.012 

hies 

-0.302 

0.245 

0.041 

-0.197 

0.1 

Exposed  Friendly 

1 

-0.017 

F  Location  Known 

-0.045 

■aERi 

#  Casualties 

1 

-0.374 

mmuM 

-0.214 

-0.305 

1 

Expected  Civilian  Actions 

-0.017 

0.281 

-0.172 

-0.007 

1 

-0.139 

0.047 

Advantage 

-0.201 

0.132 

-0.156 

-0.139 

1 

0.336 

Not  in  Communication 

-0.115 

0.085 

-0.214 

-0.03 

0.047 

0.336 

1 

Table  H-36.  Anova  table  for  SABARS  Factor  1  -  Gathering  Information 


DF 

Sum  of 

Squares 

Mean 

Square 

F- Value 

P- Value 

Experience 

1 

33.423 

33.423 

7.763 

0.008 

SCENARIO 

3 

4.806 

1.602 

0.372 

0.774 

Experience  *  SCENARIO 

3 

15.067 

5.022 

1.167 

0.332 

Residual 

48 

206.650 

4.305 
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Table  H-37.  Anova  Table  for  SABARS  Factor  2  -  Focusing  Outward  Vs.  Inward 


Sum  of  Squares 

Mean  Square 

F-Value 

Experience 

1 

5.743 

5.743 

0.108 

SCENARIO 

3 

0.325 

0.928 

Experience  *  SCENARIO 

3 

2.022 

0.674 

0.315 

0.815 

Residual 

48 

102.712 

2.140 

Table  H-38.  Anova  Table  for  SABARS  Factor  3  -  Proactively  Seeking  Key  Information 


Sum  of  Squares 

Mean  Square 

P- Value 

Experience 

1 

19.091 

19.091 

3.411 

SCENARIO 

3 

36.920 

2.199 

Experience  *  SCENARIO 

3 

19.396 

1.155 

Residual 

48 

268.635 

5.597 

Table  H-39.  Anova  Table  for  SABARS  Factor  4  -  Focusing  on  the  Big  Picture 


Experience 

6.285 

IBIMS 

SCENARIO 

3 

0.334 

Experience  *  SCENARIO 

0.501 

0.538 

0.658 

Residual 

48 

44.621 

0.930 

Table  H-40.  Anova  Table  for  PSAQ  Workload 


DF 

Sum  of  Squares 

Mean  Square 

F- Value 

P- Value 

Scenario 

1 

2.286 

2.286 

IBBll 

Experience 

1 

0.143 

0.143 

^BS^ 

Scenario  *  Experience 

1 

0.143 

0.143 

Residual 

24 

18.857 

0.786 
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Table  H-41 .  Anova  Table  for  PSAQ  Performance 


Sum  of  Squares 

Mean  Square 

F- Value 

P -Value 

Scenario 

BIB 

1.750 

1.750 

2.227 

0.149 

Experience 

0.321 

0.321 

0.529 

Scenario  *  Experience 

1 

0.321 

^beb 

BSSSfi 

bob 

Residual 

24 

18.857 

Table  H-42.  Anova  Table  for  PSAQ  SA 


Sum  of  Squares 

Mean  Square 

F- Value 

P- Value 

Scenario 

1 

2.286 

2.286 

Experience 

1 

0.143 

0.143 

IBB 

bob 

Scenario  *  Experience 

1 

1.286 

1.286 

BffiSI 

BBS 

Residual 

24 

18.000 

0.750 
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Appendix  I:  Acronyms 


AAR  -  After  Action  Review 

ANOVA  -  Analysis  of  Variance 

AO  -  Area  of  Operations 

Cl  -  Commander’s  Intent 

CO  -  Commanding  Officer 

COA  -  Course  of  Action 

FOF  -  Field  of  Fire 

FOO  -  Field  of  Observation 

LP/OP  -  Listening  Post/Observation  Post 

MOUT  -  Military  Operations  on  Urbanized  Terrain 

NBC  -  Nuclear,  Biological,  and  Chemical 

NVG  -  Night  Vision  Goggles 

0/C  -  Observer/Controller 

OCOKA-  Observation,  Cover  and  Concealment,  Obstacles,  Key  Terrain,  Avenues  of  Approach 
PSAQ  -  Post-Trial  Participant  Subjective  SA  Questionnaire 
ROE  -  Rules  of  Engagement 
SA  -  Situation  Awareness 

SABARS  -  Situation  Awareness  Behaviorally  Anchored  Rating  Scale 

SAGAT  -  Situation  Awareness  Global  Assessment  Technique 

SART  -  Situation  Awareness  Rating  Technique 

SASO  -  Stability  and  Support  Operations 

SME  -  Subject  Matter  Expert 

SSE  -  Squad  Synthetic  Environment 
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